KR102152236B1 - Camera module - Google Patents

Abstract

The camera module according to an embodiment of the present invention includes a camera unit for capturing an image, a storage unit for storing information on a search area and an image captured by the camera unit, and the captured image using information on the search area. And a processor that searches for a motion recognition object in the image and analyzes the motion of the searched motion recognition object.

Description

Camera module {CAMERA MODULE}

The present invention relates to a camera module, and more particularly, to a camera module for motion recognition.

With the development of TV or automobile related technologies, various demands for their application applications have arisen. For example, a camera built into a TV or car may recognize a user's motion and provide a service for the user's convenience.

In general, in order to recognize a user's motion, an object of motion recognition is searched and analyzed from the entire image input through a camera. For example, even when the object of motion recognition is a user's finger or eye, the user's finger or eye may be searched for from an image in which the user's whole body is photographed, and the movement of the finger or eye may be analyzed.

As described above, when searching for an object of motion recognition from the entire image every frame, the amount of information processing to find the object is large, and it may take a lot of time to find the object. In particular, when the entire image is processed even when the size of the object for motion recognition is small or the motion is fine, unnecessary time and energy may be consumed.

An object of the present invention is to provide a camera module and a motion recognition method thereof.

The camera module according to an embodiment of the present invention includes a camera unit for capturing an image, a storage unit for storing information on a search area and an image captured by the camera unit, and the captured image using information on the search area. And a processor that searches for a motion recognition object in the image and analyzes the motion of the searched motion recognition object.

The information on the search area may include location information of a reference point of the motion recognition target and location information of a search area including the reference point.

The processor may extract position information of a reference point from the searched motion recognition object, and analyze the motion of the motion recognition object using the extracted position information of the reference point.

The processor compares the location information of the reference point stored in the storage unit with the location information of the reference point extracted by the processor, and the difference between the location information of the stored reference point and the location information of the extracted reference point exceeds a threshold value Information on the search area may be changed according to the location information of the extracted reference point.

When the processor fails to search for a motion recognition target in the captured image by using the information on the search area, the processor may search for a motion recognition target from the entire area of the captured image.

The camera unit may periodically capture an image, and the processor may further include a timer for periodically searching for a motion recognition target.

The motion recognition method of the camera module according to an embodiment of the present invention includes the steps of capturing an image, searching for a motion recognition target in the captured image using information on a pre-stored search area, and motion of the searched motion recognition target It includes the step of analyzing.

The analyzing may include extracting position information of a reference point from the searched motion recognition object, and tracking a motion of the motion recognition object using the extracted position information of the reference point.

Comparing the location information of the reference point stored in advance with the location information of the extracted reference point, and when the difference between the location information of the reference point stored in advance and the location information of the extracted reference point exceeds a threshold, the extracted location information of the reference point Accordingly, the step of changing the information on the search area may be further included.

According to an embodiment of the present invention, it is possible to obtain a camera module or system for recognizing motion related to a specific object in a captured image. In particular, it is possible to obtain a camera module or system that has a small amount of information processing, a fast processing speed, and a low power consumption.

1 is a block diagram illustrating a motion recognition system including a camera module according to an embodiment of the present invention.
FIG. 2 is a flowchart illustrating a method of extracting information on a search area from an entire area by a camera module according to an embodiment of the present invention.
3 is a flowchart illustrating a method of searching for a motion recognition target in a search area by a camera module according to an embodiment of the present invention.
4 is a flowchart illustrating a method of updating information on a search area by a camera module according to an embodiment of the present invention.
5 to 8 are diagrams illustrating a search area and a reference point of a motion recognition object according to an embodiment of the present invention.

The present invention is intended to illustrate and describe specific embodiments in the drawings, as various changes may be made and various embodiments may be provided. However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms including ordinal numbers, such as second and first, may be used to describe various elements, but the elements are not limited by the terms. These terms are used only for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a second element may be referred to as a first element, and similarly, a first element may be referred to as a second element. The term and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings, but the same reference numerals are assigned to the same or corresponding components regardless of the reference numerals, and redundant descriptions thereof will be omitted.

1 is a block diagram illustrating a motion recognition system including a camera module according to an embodiment of the present invention.

Referring to FIG. 1, the motion recognition system includes a camera module 100 and an external device 200. The camera module 100 includes a camera unit 110, a storage unit 120, a processor 130, a transmission/reception unit 140, and a timer 150. The camera module 100 according to an embodiment of the present invention may be built in or connected to, for example, a TV or a car.

The camera unit 110 photographs an image. For example, the camera unit 110 is a printed circuit board (PCB), an image sensor disposed on the PCB and electrically connected to the PCB, an IR filter disposed on the image sensor and blocking near-infrared rays, an IR filter It may include a plurality of lenses that are disposed on the image and are sequentially arranged from the object side to the image side. The image sensor may be connected to the PCB by flip chip bonding or wire bonding. The image sensor may be, for example, a Charge-Coupled Device (CCD) or a Complementary Metal-Oxide Semiconductor (CMOS) sensor. The image sensor may generate an image signal by receiving light incident through a plurality of lenses. The image signal generated by the image sensor may be transmitted to an external device 200 connected by wire or wirelessly through a circuit pattern formed on a PCB and displayed as an image.

The storage unit 120 stores an image captured by the camera unit 110. In addition, the storage unit 120 may store preset information for motion recognition.

The processor 130 controls the overall operation of the camera module 100. In particular, the processor 130 may search for a motion recognition object in an image captured by the camera unit 110 using information stored in the storage 120 and analyze the motion of the searched motion recognition object.

The transceiver 140 communicates with the external device 200 by wire or wirelessly. For example, the transmission/reception unit 140 may transmit an image captured by the camera unit 110 or motion information analyzed by the processor 130 to the external device 200. In addition, the transmission/reception unit 140 may receive a signal for controlling the camera module 100 from the external device 200.

The timer 150 controls the time of the camera module 100. In particular, the timer 150 may allow the camera unit 110 to periodically capture an image or the processor 130 to periodically search for a motion recognition target.

1 illustrates that the camera unit 110, the storage unit 120, the processor 130, the transceiver 140, and the timer 150 are included in one module, but the present invention is not limited thereto. At least one of the storage unit 120, the processor 130, and the timer 150 is disposed outside the camera module, and may be connected wirelessly or wired through the transmission/reception unit 140.

Hereinafter, a method of recognizing motion by the camera module according to an embodiment of the present invention will be described in detail.

FIG. 2 is a flowchart illustrating a method of extracting information on a search area from an entire area by a camera module according to an embodiment of the present invention, and FIG. 3 is a motion diagram of a camera module according to an embodiment of the present invention. It is a flowchart illustrating a method of searching for a recognition object, and FIG. 4 is a flow chart showing a method of updating information on a search area by a camera module according to an embodiment of the present invention. 5 to 8 are diagrams illustrating a search area and a reference point of a motion recognition object according to an embodiment of the present invention.

Referring to FIG. 2, when a timer transmits a notification to the camera unit 110 at a period of T1 (S200), the camera unit 110 captures an image (S210), and the captured image is stored in the processor 130 and the storage unit. It is transmitted to 120 (S220).

The processor 130 analyzes the entire area of the image to extract a reference point of a motion recognition target (S230), and designates a search area including the reference point (S240). As illustrated in FIG. 5, when the image includes the whole body of a person and a finger gesture is to be analyzed, the finger may be a motion recognition target. In this case, the processor 130 may scan the entire area of the image for each line or pixel, extract the position of the fingertip as a reference point, and designate a search area based on the reference point. As another example, when trying to grasp the movement of the pupil, the pupil may be a motion recognition target. In this case, the processor 130 scans the entire area of the image for each line or pixel to extract the center of the pupil as a reference point, and may designate a search area based on the reference point. Here, the search area may be a predetermined area including a reference point, and may be variously set as a circle, an ellipse, a rectangle, and a triangle.

Referring back to FIG. 2, the processor 130 transmits information on the designated search area to the storage unit 120 (S250), and the storage unit 120 receives information on the search area received from the processor 130. Save (S260). Here, the information on the search area may include location information of a reference point of a motion recognition target and location information of a search area including the reference point. The location information of the reference point and the location information of the search area may be represented by coordinates, for example.

Meanwhile, as shown in FIG. 3, after time T1 has elapsed, the timer 150 transmits a notification to the camera unit 110 again (S300), and the camera unit 110 may capture an image (S310).

The camera unit 110 transmits the captured image to the processor 130 and the storage unit 120 (S320), and the storage unit 120 stores information on the search area previously stored in step S260 of FIG. 130) (S330).

The processor 130 searches for a search area among the entire area of the image (S340), and determines whether there is a motion recognition target in the search area (S350). That is, the processor 130 scans only the search area out of the total area, as shown in FIG. 6. In this way, it is not necessary to scan the entire area every frame, and by scanning only a search area that is a part of the entire area, it is possible to reduce the amount of information processing and increase the processing speed.

When there is a motion recognition target in the search area, the processor 130 analyzes the motion of the motion recognition target (S360). That is, the processor 130 extracts a reference point of the motion recognition target within the search area, and analyzes the motion by using a difference between the extracted reference point and a reference point previously stored in the storage unit 120. As an example, when a finger gesture is to be identified as shown in FIG. 7, the processor 130 extracts the coordinates of the fingertip, which is a reference point, within the search area, and compares the coordinates of the fingertip stored in advance to analyze the gesture of the finger. can do.

Meanwhile, if there is no motion recognition target in the search area in step S350, the entire area is searched and it is determined whether there is a motion recognition target in the entire area (S370). When there is a motion recognition target in the entire area, the processor 130 analyzes the motion of the motion recognition target (S360). That is, the processor 130 extracts a reference point of the motion recognition target within the entire area, and analyzes the motion by using a difference between the extracted reference point and a reference point previously stored in the storage unit 120.

When there is no motion recognition object in the entire area, the processor 130 waits for the image captured in the next cycle to be delivered.

When the time T1 elapses, the timer 150 transmits a notification to the camera unit 110 again (S380), and the camera unit 110 captures an image (S380).

Meanwhile, as shown in FIG. 8, the motion recognition target may move as time elapses. Accordingly, it is necessary to continuously update the search area.

Referring to FIG. 4, the processor 130 extracts position information of a reference point from a motion recognition object (S400), and compares the extracted position information of the reference point with the previously stored position information of the reference point (S410). In addition, if the extracted position information of the reference point and the previously stored position information of the reference point are greater than or equal to the threshold value (S420), it is determined that the motion recognition target has moved, and the information on the search target may be changed and stored (S430).

For example, as shown in FIG. 8, when a motion recognition target is not searched within a stored search area, the processor 130 may search for a motion recognition target for the entire area. When a motion recognition target is searched within the entire area, the processor 130 extracts a reference point from the motion recognition target. Then, the extracted reference point is compared with the stored reference point. When the location information of the extracted reference point and the stored reference point differs by more than a predetermined level, the processor 130 determines that the motion recognition target has moved, and may redesignate the search area based on the extracted reference point. The re-designated search region is stored in the storage unit 120, and a re-designated search region may be searched for an image captured thereafter. As shown in FIG. 7, even when a motion recognition object is searched within the stored search area, the processor 130 extracts a reference point from the motion recognition object, and when the extracted reference point and the stored reference point position information differ by more than a predetermined level, The search area can be redesignated based on the extracted reference point.

Although described above with reference to preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. You will understand that you can do it.

100: camera module
110: camera unit
120: storage unit
130: processor
140: transceiver unit
150: timer

Claims (9)

A camera unit that shoots video,
A storage unit for storing information on a search area and an image captured by the camera unit,
And a processor that searches for a motion recognition target in the captured image using information on the search area and analyzes the motion of the searched motion recognition target,
The size of the search area is larger than the size of the reference point of the motion recognition target, and is smaller than the size of the entire area of the captured image,
The information on the search area includes location information of a reference point of the motion recognition target and location information of a search area including the reference point,
The information on the search area includes location information of a reference point of the motion recognition target and location information of a search area including the reference point,
The processor compares the location information of the reference point stored in the storage unit with the location information of the reference point extracted by the processor, and when the difference between the location information of the stored reference point and the location information of the extracted reference point exceeds a threshold value Change information on the search area according to the location information of the extracted reference point,
The camera module further comprises a timer for the camera unit to periodically capture an image and for the processor to periodically search for a motion recognition target. delete delete delete The method of claim 1,
When the processor fails to search for a motion recognition target in the captured image using information on the search area, the camera module searches for a motion recognition target from the entire area of the captured image. delete In the camera module motion recognition method,
Taking an image,
Searching for a motion recognition target in the captured image by using information on the pre-stored search area, and
Analyzing the motion of the searched motion recognition object,
The analyzing step,
Extracting the location information of the reference point from the searched motion recognition object, and
Including the step of tracking the motion of the motion recognition target using the extracted position information of the reference point,
Comparing the location information of the reference point stored in advance with the location information of the extracted reference point, and,
If the difference between the location information of the previously stored reference point and the location information of the extracted reference point exceeds a threshold value, changing the information on the search area according to the location information of the extracted reference point,
The step of photographing, the step of searching, the step of analyzing, and the step of changing are performed periodically. delete delete

Images