WO2020116677A1 - Motion recognition device control using captured image, and motion recognition method, server, and program using same - Google Patents

Abstract

The present invention relates to motion recognition device control using a captured image, and a motion recognition method using same, and analyzes an image obtained by photographing both a target image and the motion recognition device so as to calculate the real size of the target image, and scans a user touch on the target image through the motion recognition device so as to analyze the touched region, thereby enabling the provision of various kinds of information to a user.

Description

Control of motion recognition devices using captured images and motion recognition methods, servers and programs using them

The present invention relates to control of a motion recognition device using a captured image and a motion recognition method using the motion recognition device.

A motion recognition device means a device that moves objects or people's movements digitally using sensors. Motion recognition is classified into mechanical, magnetic, and optical methods.

On the other hand, light detection and ranging (LiDAR) is a device that measures the distance or atmospheric phenomena by using a reflector or scatterer to emit pulsed laser light into the atmosphere, and is also called a laser radar. The time measurement of reflected light is calculated by clock pulse, and its frequency is 5 MHz at 30 MHz and 1 m at 150 MHz, so it shows superior detection efficiency than conventional cameras and sensors.

Therefore, when the principle of lidar is applied to a motion recognition device, the detection efficiency of the motion is increased, so that precise motion recognition will be possible.

In addition, no attempt has been made to attach/install a motion recognition device to a general image and provide various information as the user touches the image.

When this technology is applied, it can be applied to various images such as subway, bus route maps, advertisements, movies, concert posters, etc., so that various information not included in the image can be provided to the user, but related documents are currently It has not been released.

The present invention for solving the above-described problems is to analyze the image captured by the image and the motion recognition device together, calculate the actual size of the image, control of the motion recognition device using the captured image and the motion recognition method using the same Can provide.

In addition, according to the present invention, by extracting the object images included in the image and storing the coordinates of each object image, the user's input information scanned by the motion recognition device can recognize the corresponding object image, using a captured image. It is possible to provide a motion recognition device control and a motion recognition method using the same.

In addition, the present invention can provide a method for controlling a motion recognition device using a captured image and a motion recognition method using the same, which provides various information corresponding to an object image touched by the user to the user.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

Control of a motion recognition device using a captured image and a motion recognition method using the same according to an embodiment of the present invention for solving the above-described problems are methods performed by a computer, attached or installed on one side of the image and the image A motion recognition device (Lidar) receiving an image captured together; Calculating an actual size of the image based on the actual size of the video and the motion recognition device; And receiving identification information of the image.

In addition, extracting one or more object images including pictures or characters included in the image, and storing coordinates on the image of each object image; And receiving tag information or content information of each extracted object image.

In addition, receiving input information including a user's touch or gesture on the image scanned through the motion recognition device, and recognizing at least one object image located at coordinates corresponding to the input information; Extracting at least one object information through tag information or content information of the recognized object image; And providing the extracted object information through at least one of a display device, a voice output device, and a user terminal provided at a location where the image is located. The motion recognition device further includes rotating the reflector to It is to obtain coordinates of input information including a user's touch or gesture on an image.

In addition, as the step of receiving the number information scanned through the motion recognition device, the number information is sequentially drawn on the image from the user, receiving the number information; And identifying a terminal matching the numeric information as a user terminal.

In addition, receiving input information of the first gesture of the user scanned through the motion recognition device; And providing the object information to the identified user terminal.

Further, in the information extraction step, when the recognized object image is a category image representing identification information of the image, at least one of representative information, real-time information, and announcement information of the image is extracted as object information.

Further, in the information extraction step, when the user's input information is a touch input to an image of an object, the content information allocated to the object image is extracted as object information, or based on the identification information of the image By matching the tag information of the object image, the matched information is extracted as the object information.

In addition, in the information extraction step, when the input information of the user is a touch and drag input to the second object image from the first object image, the first object image and the second object image are based on the identification information of the image. Matching related information (relational information) by extracting the related information (relational information) between the contents information of the object information as object information, or searching the tag information of the first object image and the second object image in real time based on the identification information of the image Is extracted as object information.

In addition, in the information extraction step, when the input information of the user is a range input including two or more object images, the content of two or more object images included in the range input based on the identification information of the image Common information between information is extracted as object information, or tag information of two or more object images included in the range input is searched in real time based on the identification information of the image to extract matched common information as object information.

Also, based on the actual size of the calculated image, setting a scanning range of the motion recognition device; may include.

Control of a motion recognition device using a captured image and a motion recognition server using the same according to an embodiment of the present invention for solving the above-described problem include an image and a motion recognition device (Lidar) attached or installed on one side of the image. A receiving unit which receives the photographed image and receives identification information of the image; And a calculator configured to analyze the image based on the actual size of the pre-motion recognition device and calculate the actual size of the image.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium for recording a computer program for executing the method may be further provided.

The control of the motion recognition device using the captured image and the motion recognition program using the same according to another embodiment of the present invention is combined with a computer that is a hardware to control the motion recognition device using the above-mentioned captured image and the motion recognition method using the same. Run and stored on the medium.

According to the present invention as described above, there is an effect that can calculate the actual size of the image based on the actual size of the image and the motion recognition device photographed together with the image and motion recognition device.

In addition, according to the present invention, through the scanning of the coordinates of each object image included in the image and the motion recognition device, there is an effect of recognizing the location touched by the user's touch input on the general image.

In addition, according to the present invention, there is an effect of providing various information corresponding to the object image touched by the user.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a flowchart of a motion recognition device using a captured image and a motion recognition method using the same according to an embodiment of the present invention.

2 is a diagram illustrating a route map of the Seoul Subway Line 2 as an image according to an embodiment of the present invention.

FIG. 3 is a diagram showing object images included in the image of FIG. 2.

FIG. 4 is a diagram illustrating that a user touches a category image in the image of FIG. 2.

FIG. 5 is a diagram illustrating that a user performs a touch and drag input on the image of FIG. 2.

FIG. 6 is a diagram illustrating that the user has performed range input in the image of FIG. 2.

FIG. 7 is a diagram illustrating that a user performs a first gesture input in the image of FIG. 2.

FIG. 8 is a diagram illustrating that a user performs a touch input of drawing a number on the image of FIG. 2.

9 is a view illustrating a movie poster as an image according to an embodiment of the present invention.

FIG. 10 is a diagram illustrating that a user touches the object image of the trailer view in the image of FIG. 9.

11 is a block diagram of a motion recognition device using a captured image and a motion recognition server using the same.

Advantages and features of the present invention, and methods for achieving them will be clarified with reference to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the embodiments allow the disclosure of the present invention to be complete, and are common in the technical field to which the present invention pertains. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing the embodiments and is not intended to limit the present invention. In the present specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components other than the components mentioned. The same reference numerals throughout the specification refer to the same components, and “and/or” includes each and every combination of one or more of the components mentioned. Although "first", "second", etc. are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings commonly understood by those skilled in the art to which the present invention pertains. In addition, terms defined in the commonly used dictionary are not ideally or excessively interpreted unless explicitly defined.

The motion recognition device 20 may recognize the motion of the subject by obtaining the distance and coordinates of the subject.

Various conventional techniques can be used as the motion recognition device 20 of the present invention. However, it may be desirable to use a lidar as the motion recognition device 20 of the present invention.

Light detection and ranging (LiDAR) uses a light source, such as a laser, and measures the time from reflection from the subject to return to the sensor to obtain the distance and coordinates of the subject.

On the other hand, as the motion recognition device 20 of the present invention, general non-laser light can be emitted. Since the motion recognition device 20 of the present invention is frequently used for obtaining a distance and a coordinate of a short-distance object, even when a general light emitting diode (LED) is used instead of a laser having a high light density, there is a problem in scanning a short range. This is because it is possible to perform precise tracking rather than laser light.

In addition, the motion recognition device 20 according to an embodiment of the present invention may perform tracking by applying a VCSEL or a laser as well as an LED as a light source.

As a typical example, a motion recognition device 20 is installed in a display device such as a monitor or a TV without a touch panel, so that an effect such as a touch function is inserted into the corresponding monitor or TV is exerted.

As described above, the motion recognition device 20 is utilized in various fields, but technologies such as attaching/installing the motion recognition device 20 to a general image and providing various information as the user touches the image are disclosed. It is not a situation.

Applicants have applied the motion recognition device 20 to a general image, a picture, not a display device, by utilizing the advantages of the motion recognition device 20, and are not included in the image as the user touches the image or picture. The present invention has been devised to provide various information.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flow chart of a method of controlling a motion recognition device using a captured image and a motion recognition method using the same according to an embodiment of the present invention, and FIGS. 2 to 8 are route maps of the Seoul Subway Line 2 to illustrate an embodiment of the present invention It is a diagram illustrating various situations using images.

Hereinafter, control of a motion recognition device using a captured image and a motion recognition method using the same will be described with reference to FIGS. 1 to 8.

Control of a motion recognition device using a captured image according to an embodiment of the present invention and a motion recognition method using the same are performed by a computer, and in the following embodiment, it will be described as being performed mainly by the server 100.

In addition, in the embodiment of the present invention, providing the user terminal 300 may be interpreted as meaning provided through a service application installed in the user terminal 300.

First, the reception unit 110 of the server 100 receives an image photographed together by an image 200 and a motion recognition device (Lidar, 20) attached or installed on one side of the image 200 from the terminal 300. (Step S500)

Preferably, the reception unit 110 of the server 100 receives an image, a picture taken by the motion recognition device 20 and the image 200 together as shown in FIG. 2 by the administrator using the photographing means of the terminal 300. do.

At this time, the administrator who took the picture may directly send the picture to the server 100 through the terminal 300, and the service application itself provided by the server 100 provides a shooting mode to proceed with the shooting and the captured image , The picture may be automatically sent to the server.

In addition, although the server 100 may provide a method and a guide for photographing an image with the terminal 300 of the administrator, it is applicable to any image or picture that includes the motion recognition device 20 and the image 200. Therefore, the above items can be selectively employed.

Next to step S500, the calculator 120 of the server 100 calculates the actual size of the image 200 based on the actual size of the image and motion recognition device 20 received in step S500. (Step S510)

As an example of this, a proportional expression of " size of the image 200 in the captured image: size of the motion recognition device 20 in the captured image = size of the actual image 200: size of the actual motion recognition device 20 " is used. By doing so, it means calculating the size of the actual image 200.

In addition to the proportionality exemplified above, any method that can calculate the actual size of the image 200 based on the actual size of the captured image and the motion recognition device 20 may be applied.

2 is a diagram illustrating a route map of the Seoul Subway Line 2 with the image 200 according to an embodiment of the present invention. (For reference, only some stations on subway line 2 are shown for convenience of explanation)

For example, when the administrator photographs an image (motion map 200, Seoul Subway Line 2 route) with the motion recognition device 20 attached to the top through the terminal 300 as shown in FIG. 2, the calculator 120 moves Based on the actual size of the recognition device 20, the actual size of the image 200 (Route map of Seoul Subway Line 2) can be calculated.

In addition, the calculating unit 120 may further include setting a scanning range of the motion recognition device 20 based on the actual size of the calculated image 200.

For example, when the setting unit 180 of the server 100 is 1000 mm x 1000 mm in size, it may mean setting the scanning range so that the motion recognition device 20 can acquire an optical signal only in the corresponding range. have.

For another example, when the setting unit 180 of the server 100 is 1000 mm x 1000 mm in size, it means that the motion recognition device 20 sets the scanning range to scan only the corresponding range.

Through this, it is possible to prevent unnecessary motions by scanning the portion where the motion recognition device 20 is not needed.

Next, in step S510, the reception unit 110 of the server 100 receives identification information of the image 200. (Step S520)

It means that the identification information of the corresponding image 200 is input from an administrator who photographed the image 200 or another administrator.

More specifically, the image 200 may be applied with an image having a specific purpose, for example, a subway map, a bus map, a movie poster, a museum poster, a concert poster, or an advertising poster.

Therefore, subways, buses, movies, museums, concerts, advertisements, etc. may be identification information of the corresponding image 200, and such identification information may be a base for retrieving various information according to user input information in the future.

In addition, a plurality of identification information may be stored instead of one identification information. For example, "public transportation" may be stored together as identification information along with a subway or a bus, or "cultural life" together with a movie. Words representing the image 200 may be stored together as identification information.

In addition, the identification number of the corresponding image 200 is stored together with the identification information, so that the place information to which each image 200 is attached and provided may be identified, and each image 200 may be distinguished.

2, for example, the reception unit 110 of the server 100 may receive “Seoul Subway Line 2 Route Map” as identification information.

Next, in step S520, the extraction unit 130 of the server 100 extracts one or more object images 220 including pictures or characters included in the image 200, and the images of each object image 220 The coordinates on (200) are stored. (Step S530)

3 is a diagram showing object images 220 included in the image 200 of FIG. 2.

The extracting unit 130 of the server 100 analyzes the received image 200 as shown in FIG. 2, and one or more object images including pictures or characters included in the image 200 (Route Map of Seoul Subway Line 2) 220), and means to store coordinates on the image 200 of each object image 220.

Therefore, as shown in FIG. 3, "Seoul Subway Line 2 Route Map", "Hapjeong", Hongdae Entrance, "Sinchon", "Ewha University", "Ahyeon", "Chungjeongro", "City Hall", "Euljiro Entrance", "Euljiro 3 It can be seen that the object images 220 of "A" and "Euljiro 4-Ga" are extracted.

At this time, the image 200 analysis module of the server 100 analyzes the image 200 to grasp the relationship between the picture and the character, and can automatically set the range of each object image 220, and the administrator re-adjusts the error Can be modified.

As described above, when the object image 220 is extracted through the extraction unit, coordinates on the image 200 of each object image 220 may be stored. For example, various methods are applicable, such as setting the coordinates by starting the lower left corner of the image 200 with coordinates of (X=0, Y=0).

Next to step S530, the storage unit 140 of the server 100 receives tag information or content information of each object image 220 extracted in step S520. (Step S540)

Each object image 220 on the image 200 is extracted through steps S510 to S530, and it means that information of each extracted object image 220 is input as shown in FIG. 3 through step S540.

More specifically, as the tag information of the object image 220, the name, description, summary, etc. of the object image 220 may be stored.

For example, in the object image 220 corresponding to "Hongik University Entrance", Hongik University Entrance, Hongik University, Line 2, Transit Station, Mapo-gu, Donggyo-dong, etc. may be stored as tag information.

In addition, tag information such as Seoul City Hall, Ewha Womans University, Jung-gu, Jeong-dong, Deoksugung, Deoksu Elementary School, Seoul Museum of Art, and Seoul Plaza can be stored in the object image 220 corresponding to "City Hall Station".

As with the tag information exemplified through the two object images 220, information such as the name, summary, location, surrounding landmarks, and tourist attractions of the object image 220 may be input.

However, the above example is because the identification information of the image 200 corresponding to the object image 220 is a subway map, and when the identification information is different, other tag information may be stored.

In addition, the content information may correspond to image data, audio data, and Internet sites corresponding to the object image 220.

For example, when the image 200 is a movie poster and the object image 220 is a "trailer", the trailer movie data of the corresponding movie may be input as content information of the "trailer" object image 220.

As described above, the setting of the image 200 and the object image 220 is completed through steps S500 to S540, and it will be described below that various information is provided according to user input information.

The recognition unit 150 of the server 100 receives the user's input information on the image 200 scanned through the motion recognition device 20, and at least one object image 220 located at coordinates corresponding to the input information ). (Step S550)

The input information includes a user's touch or gesture. For example, since a specific object image 220 may be touched as clicking, or a touch and drag input may be performed from the first object image 220 to the second object image 220, various inputs may be included. have.

Since the actual size of the image 200 was calculated in step S510 above, and the coordinate information of the object images 220 included in the image 200 was stored in step S530, the recognition unit 150 of the server 100 When the user's input information is received through the motion recognition device 20, the user can recognize which object image 220 has been input.

Next to step S550, the information extraction unit 160 of the server 100 extracts at least one object information through tag information or content information of the object image 220 recognized in step S550. (Step S560)

Then, after step S560, the information providing unit 170 provides the object information extracted in step S560 through at least one of a display device, a voice output device, and a user terminal 300 provided in a location where the image 200 is located. do. (Step S570)

More specifically, various cases may occur depending on the user's input information, which will be described in detail below.

First case>

FIG. 4 is a diagram illustrating that a user touches the category image 210 in the image 200 of FIG. 2.

In an embodiment of the present invention, the image 200 includes one or more object images 220 and a category image 210. Preferably, at least one of the object images 220 may correspond to the category image 210.

Referring to FIG. 4, the user touches the category image 210 of “Seoul Subway Line 2 Route Map” in the image 200.

At this time, the category image 210 is also one of the object images 220, but different from other object images 220, it corresponds to the object image 220 representing the image 200.

For example, if the image 200 corresponds to a subway map, the subway line 2, line 5, etc. may be applied.

As another example, when the image 200 corresponds to a bus route map, it may be a bus number.

As another example, if the image 200 corresponds to a movie poster, it may be a movie title.

As such, the category image 210 means an object image 220 representing the corresponding image 200.

In the information extraction step (step S560), when the information extraction unit 160 of the server 100 is the category image 210 that represents the identification information of the image 200, the object image 220 recognized in step S560, At least one of representative information, real-time information, and announcement information of the image 200 is extracted as object information.

Accordingly, when the category image 210 is touched as shown in FIG. 4, it can be seen that information on the Seoul Subway Line 2 is extracted and provided as shown in FIG. 4.

Seoul Subway Line 2 is a circular line, transferable lines 1 to 9, and real-time information is being extracted from Jamsil Station, indicating that the operation is being delayed due to a malfunction. In addition, when there are various notices, it may be extracted and provided as object information.

As described above, various information that is not provided on the image 200 of the subway map is provided to exert the effect of providing various information to the user.

Case 2>

In the information extraction step (S560), the information extraction unit 160 of the server 100, when the user's input information is a touch input to one object image 220, the content allocated to the object image 220 Information is extracted as object information.

In addition, if there is no content information allocated to the corresponding object image 220, the information extraction unit 160 searches and matches the tag information of the corresponding object image 220 based on the identification information of the image 200. Information is extracted as object information.

For example, when the user touches the “Hongik University entrance” object image 220 in the image 200, the subway line 2 corresponding to the identification information of the image 200 and the Hongik University entrance corresponding to the tag information of the Hongik University entrance , Hongik University, Line 2, Transit Station, Mapo-gu, Donggyo-dong, and related information are extracted as object information.

Also, the information extraction unit 160 may extract object information in consideration of time information when the user touches the image 200.

For example, time information of a next train arriving at Hongik University Station, subway timetable of Hongik University Station, landmarks at Hongik University Station, tourist information, and information of Hongik University can be matched and extracted as object information.

At this time, the extracted and matched information may be implemented in various ways, such as identification information of the image 200 and information having the highest matching level may be selected, or information suitable for a time zone touched by a user may be selected.

Also, if content information is stored in the corresponding object image 220, the content information may be extracted and provided as object information.

For example, content information such as a map image of a walking path from Hongik University Station to Hongik University and a roadview image around Hongik University Station may be applied.

In addition, the information collection unit collects various reviews and feedbacks of users, classifies and stores useful or unnecessary information for the user, thereby learning the information extraction unit 160 to improve object information extraction.

Case 3>

In the information extraction step (step S560), the information extraction unit 160 of the server 100, when the user's input information is a touch and drag input to the second object image 240 from the first object image 230, Based on the identification information of the image 200, association information (relational information) between the content information of the first object image 230 and the second object image 240 is extracted as object information.

In addition, if there is no content information allocated to the corresponding object image 220, the information extraction unit 160 based on the identification information of the image 200, the first object image 230 and the second object image 240 Real-time search of tag information of and extracts matched related information (relational information) as object information.

FIG. 5 is a diagram illustrating that a user performs a touch and drag input on the image 200 of FIG. 2.

For example, if the user touches the "happ" object image 220 in the image 200 and drags it to the "viewing" object image 220, the "happ" corresponds to the first object image 230, " The viewing" corresponds to the second object image 240.

In the same way as in the second case, the information extracting unit 160 may extract object information in consideration of time information when the user touches the image 200.

Accordingly, the information extraction unit 160 searches the subway line 2 corresponding to the identification information of the image 200, the tag information of the "happ" object image 220, and the tag information of the "viewing" object image 220 in real time. Then, the matched related information (relational information) is extracted as object information.

In this case, the related information (relational information) means related information and related information between two object images 220.

More specifically, it does not extract the information of each of the two object images 220 into the object image 220, but means to derive common information, differences, etc. between the two object images 220.

Fig. 5 shows the example.

5, since the user performed a touch and drag input from Hapjeong to the viewing direction, 6 stops from Hapjeong to viewing, and the object information required to take about 13 minutes was derived and provided.

In addition, the time information touched from the user is also considered, and it can be seen that the next train time arriving at Hapjeong Station is extracted and provided as object information.

In addition, as illustrated in FIG. 5, information such as tourist attractions, restaurants, and landmarks between Hapjeong Station and City Hall Station may also be extracted and provided as object information.

In addition, if content information is stored in the corresponding object images 220, content information may be extracted and provided as object information.

For example, since the user is expected to get off at City Hall, a video of landmarks around Deoksugung Station (Deoksugung Palace, Seoul Plaza, etc.), a map image showing the walking path from Seoul Station to Seoul City Hall (200), and roadview around Seoul Station An image 200 or the like may be applied.

In addition, various tourism promotional images provided by the city of Seoul may be stored as content information and extracted and provided as object images.

Case 4>

In the information extraction step (step S560), the information extraction unit 160 of the server 100 identifies the corresponding image 200 when the user's input information is a range-specified input including two or more object images 220 Based on the information, common information between content information of two or more object images 220 included in the above-described range input and extraction is extracted as object information.

In addition, the information extraction unit 160 retrieves the tag information of two or more object images 220 included in the range input based on the identification information of the corresponding image 200 in real time to match the common information as the object information. Can be extracted.

FIG. 6 is a diagram illustrating that a user performs a range input in the image 200 of FIG. 2.

As illustrated in FIG. 6, when the user performs a range designation input including the “hapjeong”, “hongdae entrance”, and “sinchon” object images 220, three object images 220 are included.

Accordingly, the information extraction unit 160 extracts common information between the content information of the three object images 220 as object information based on the identification information of the image 200, or searches the tag information in real time to match the matched common information. Can be extracted as object information.

For example, common information of the three object images 220 may be extracted as object information and provided as shown in FIG. 6.

Since all three are located near the university district, there are many shopping streets, restaurants, etc., it is possible to provide information about restaurants, information on various tourist attractions, and schedules of various events in nearby areas.

When a user visits for tourist purposes, it is easy to visit a specific place as the above information is provided, and you can access the latest information on the area or participate in various events if the schedule meets.

These functions are those that could not be implemented at all in the subway map image 200 that previously provided only route information, and have a great effect on tourists.

In addition, if the content information is stored as in other cases, the stored content information may be searched and content information having a common point match may be extracted and provided as object information.

The information providing unit 170 displays the extracted object information according to the case of the first case, second case, third case, and fourth case in the display device, voice output device, and It may be provided through at least one of the user terminal 300.

At this time, the server 100 may need a process of identifying the user terminal 300, and various methods may be applied as follows.

The identification number of the corresponding image 200 is described on one side of the image 200, and the server 100 allows the user terminal 300 to execute the service application installed on the terminal 300 and input the corresponding identification number. Can be identified.

As another example, the QR code (Quick Response Code) is included on one side of the image 200, and the server 100 is executed by the user running a service application installed on the terminal 300 and scanning the QR code through the photographing unit. The user terminal 300 may be identified.

As another example, the server 100 may identify the user terminal 300 as NFC is installed/attached to one side of the image 200 and recognizes NFC through the user terminal 300.

Although the above methods can also be implemented, there is a hassle of entering an identification number, attaching a QR code, or installing NFC, so that the user terminal 300 can also be identified through the motion recognition device 20 as shown below. Can be.

Specifically, the receiving unit 110 of the server 100 receives the numerical information scanned through the motion recognition device 20. At this time, the numeric information means that it is sequentially drawn on the image 200 from the user.

In addition, the identification unit of the server 100 identifies the terminal 300 that matches the numeric information to the user terminal 300 and provides object information.

FIG. 8 is a diagram illustrating that a user performs a touch input of drawing a number on the image 200 of FIG. 2.

As shown in FIG. 8, when the user draws a number of the user terminal 300 on the image 200 with a finger (a tool such as a pen may be described using a finger as a typical example), the motion recognition device 20 It scans it and recognizes that the user has entered the numbers 010-1234-5678 sequentially.

And, since the user terminal 300 has been identified, the information providing unit 170 of the server 100 provides object information to the terminal 300 corresponding to the number 010-1234-5678, so that the user can receive the information. To make.

In addition, in FIG. 8, the user draws and touches 11 digits as arranged in order, but the motion recognition device 20 can scan even when overlapping, so if the order of the numbers is correct, the touch method and position are not limited. Does not.

As described above, since another separate device is not installed to identify the user's terminal 300, the motion recognition device 20 scans the user's input information so that the user's terminal 300 can be identified. In addition, it has a remarkable effect in terms of cost and provides convenience to the user.

In one embodiment, when a user touches a specific object image 220 and inputs a first gesture on the image 200, there is a step of providing information of the corresponding object image 220 to the user terminal 300. It may be further included.

More specifically, when the receiving unit 110 of the server 100 receives the input information of the first gesture of the user scanned through the motion recognition device 20, the information providing unit 170 is the object image 220 The object information of is provided to the identified user terminal 300.

FIG. 7 is a diagram illustrating that a user performs a first gesture input in the image 200 of FIG. 2.

7 illustrates that a question mark (?) is drawn by touch as a first gesture. In this way, by using symbols commonly used by people around the world, such as question marks, foreigners who do not know the language of the region can easily request information.

9 is a diagram illustrating a movie poster with an image 200 according to an embodiment of the present invention, and FIG. 10 illustrates that a user touches the object image 220 of the trailer view in the image 200 of FIG. 9 It is a drawing.

The subway route map is described as an embodiment through FIGS. 2 to 8, and a case where it is applied to a movie poster through FIGS. 9 and 10 will be further described.

When the server 100 receives an image in which the motion recognition device 20 and the image 200 (movie poster) are captured, the actual size of the image 200 (movie poster) is calculated, and the object image included in the movie poster The 220s are extracted and coordinates on the image 200 of each object image 220 are stored.

Then, tag information or content information of each extracted object image 220 is received.

In the case of a movie poster, the movie title corresponds to the category image 210.

And, as shown in FIG. 9, the characters, director's name, main name, supporting role name, trailer view, advance reservation, etc., respectively, drawn on the movie poster are extracted as the object images 220, respectively.

In addition, the person shape on the image (200, movie poster) means the main characters BBB and CCC, respectively.

As an example of content information and tag information, in the object image 220 "director name", the director's name, the director's work, and the box office work can be stored as tag information, and the director's picture can be stored as content information. have.

In addition, in the object image 220 "main star", the name of the main star, the starring work, the box office work, etc. may be stored as tag information, and the picture, the main scene, and the like of the starring may be stored as content information.

Also, one or more trailer videos may be stored as content information in the object image 220 "view trailer".

In addition, content information such as a reservation method, a reservation status, and a reservation site may be stored in the object image 220 "Reservation".

Therefore, as shown in FIG. 10, when the user touches and selects the “view trailer” of the object image 220, the information providing unit 170 of the server 100 may extract and provide the trailer image of the corresponding movie as object information.

In addition, when the user touches and selects the "reservation" of the object image 220, the information providing unit 170 of the server 100 may allow the service application of the user terminal 300 to be connected to the reservation site of the corresponding movie.

In one embodiment, if a user performs a range input including both the leading BBB and the CCC, the information extraction unit 160 of the server 100 displays the tag information of the leading BBB and CCC included in the range specifying input in real time. By searching, the matched common information can be extracted as object information.

As an example of this, information such as a movie in which the leading BBB and CCC appeared together may be extracted and provided as object information.

As described above, the control of the motion recognition device 20 using the captured image according to the embodiment of the present invention and the motion recognition method using the same may be applied to the image 200 and a touch function may be implemented. 200) can provide various information that cannot be included on the image, and can be applied to various fields.

11 is a block diagram of a motion recognition server 100 using the captured image and controlling the motion recognition device 20.

However, in some embodiments, the server 100 may include fewer components or more components than the components illustrated in FIG. 11.

Referring to FIG. 11, the control of the motion recognition device 20 using the captured image and the motion recognition server 100 using the motion image according to an embodiment of the present invention include a reception unit 110, a calculation unit 120, and an extraction unit ( 130, a storage unit 140, a recognition unit 150, an information extraction unit 160, and an information providing unit 170.

The receiver 110 receives an image captured by the image 200 and a motion recognition device 20 (Lidar) attached or installed on one side of the image 200 and receives identification information of the image 200.

The calculator 120 analyzes the image received by the receiver 110 based on the actual size of the motion recognition device 20 to calculate the actual size of the image 200.

In addition, the calculator 120 may set a scanning range of the motion recognition device 20 based on the actual size of the calculated image 200.

The extracting unit 130 extracts one or more object images 220 including pictures or characters included in the image 200.

Then, the storage unit 140 stores the coordinates on the image 200 of each object image 220 extracted from the extraction unit 130.

The recognition unit 150 receives input information of the user on the image 200 scanned through the motion recognition device 20 and recognizes at least one object image 220 positioned at coordinates corresponding to the input information.

The information extraction unit 160 extracts at least one object information through tag information or content information of the object image 220 recognized through the recognition unit 150.

The information providing unit 170 provides object information extracted from the information extraction unit 160 through at least one of a display device, a voice output device, and a user terminal 300 provided in a location where the image 200 is located.

Control of the motion recognition device using the captured image according to the embodiment of the present invention described above and the motion recognition server using the same are different from the method and the category of the invention described with reference to FIGS. 1 to 10, and are duplicated. Examples are omitted.

In the above, the control of the motion recognition device using the captured image and the motion recognition method using the same according to an embodiment of the present invention described above are implemented as a program (or application) to be executed in combination with a server that is hardware and stored in a medium. Can be.

The above-described program is C, C++, JAVA, machine language, etc., in which a processor (CPU) of the computer can be read through a device interface of the computer in order for the computer to read the program and execute the methods implemented as a program. It may include a code (Code) coded in the computer language of. Such code may include functional code related to a function defining functions required for executing the above methods, and control code related to an execution procedure necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, the code may further include a memory reference-related code as to which location (address address) of the computer's internal or external memory should be referred to additional information or media necessary for the computer's processor to perform the functions. have. In addition, when the processor of the computer needs to communicate with any other computer or server in the remote to execute the functions, the code can be used to communicate with any other computer or server in the remote using the communication module of the computer. It may further include a communication-related code for whether to communicate, what information or media to transmit and receive during communication, and the like.

The storage medium refers to a medium that stores data semi-permanently and that can be read by a device, rather than a medium that stores data for a short time, such as registers, caches, and memory. Specifically, examples of the medium to be stored include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program may be stored in various recording media on various servers that the computer can access or various recording media on the computer of the user. In addition, the medium is distributed in a computer system connected by a network, so that the computer readable code may be stored in a distributed manner.

The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, a software module executed by hardware, or a combination thereof. The software modules may include Random Access Memory (RAM), Read Only Memory (ROM), Erasable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), Flash Memory, Hard Disk, Removable Disk, CD-ROM, or It may reside on any type of computer readable recording medium well known in the art.

The embodiments of the present invention have been described above with reference to the accompanying drawings, but a person skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing its technical spirit or essential features. You will understand. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and not restrictive.

[Description of Drawing Codes]

20: motion recognition device 100: server

110: receiving unit 120: calculating unit

130: extraction unit 140: storage unit

150: recognition unit 160: information extraction unit

170: information providing unit 200: image

210: Category image 220: Object image

300: user terminal

Claims (12)

1. In a way that is performed by a computer,

   Receiving an image captured by an image and a motion recognition device attached or installed on one side of the image;

   Calculating an actual size of the image based on the actual size of the video and the motion recognition device; And

   Including the step of receiving the identification information of the image;

   Control of motion recognition device using captured images and motion recognition method using the same.
2. According to claim 1,

   Extracting one or more object images including pictures or characters included in the image, and storing coordinates on the image of each object image; And

   Including tag information or content information of each of the extracted object image; includes,

   Control of motion recognition device using captured images and motion recognition method using the same.
3. According to claim 2,

   Receiving input information including a user's touch or gesture on the image scanned through the motion recognition device, and recognizing at least one object image located at coordinates corresponding to the input information;

   Extracting at least one object information through tag information or content information of the recognized object image; And

   Further comprising the step of providing the extracted object information through at least one of a display device, a voice output device, and a user terminal provided in a location where the image is located, further comprising,

   The motion recognition device, by rotating the reflector to obtain the coordinates for the input information including the user's touch or gesture on the image,

   Control of motion recognition device using captured images and motion recognition method using the same.
4. According to claim 3,

   Receiving numeric information scanned through the motion recognition device, wherein the numeric information is sequentially drawn on the image from a user; And

   Further comprising the step of identifying the terminal matching the numeric information as a user terminal;

   Control of motion recognition device using captured images and motion recognition method using the same.
5. The method of claim 4,

   Receiving input information of a first gesture of a user scanned through the motion recognition device; And

   Providing the object information to the identified user terminal; further comprising,

   Control of motion recognition device using captured images and motion recognition method using the same.
6. According to claim 3,

   The information extraction step,

   When the recognized object image is a category image indicating identification information of the image,

   At least one of representative information, real-time information, and announcement information of the image is extracted as object information.

   Control of motion recognition device using captured images and motion recognition method using the same.
7. According to claim 3,

   The information extraction step,

   When the user's input information is a touch input to the image of one object,

   To extract the content information assigned to the object image as object information, or to search the tag information of the object image based on the identification information of the image and extract matched information as object information.

   Control of motion recognition device using captured images and motion recognition method using the same.
8. According to claim 3,

   The information extraction step,

   When the user's input information is a touch and drag input from the first object image to the second object image,

   Based on the identification information of the image, the related information between the content information of the first object image and the second object image is extracted as object information,

   Based on the identification information of the image, the tag information of the first object image and the second object image is searched in real time to extract matched related information (relational information) as object information.

   Control of motion recognition device using captured images and motion recognition method using the same.
9. According to claim 3,

   The information extraction step,

   When the input information of the user is a range-specified input including two or more object images,

   Based on the identification information of the image, common information between the content information of two or more object images included in the range input is extracted as object information,

   Based on the identification information of the image, the tag information of two or more object images included in the range input is searched in real time to extract matched common information as object information.

   Control of motion recognition device using captured images and motion recognition method using the same.
10. According to claim 1,

    Including the actual size of the calculated image, setting a scanning range of the motion recognition device; includes,

    Control of motion recognition device using captured images and motion recognition method using the same.
11. A receiver configured to receive an image photographed together with an image and a motion recognition device (Lidar) attached or installed on one side of the image, and receiving identification information of the image; And

    Includes; calculating unit for analyzing the image based on the actual size of the motion recognition device, to calculate the actual size of the image;

    Control of motion recognition device using captured images and motion recognition server using the same.
12. Combined with a hardware computer, a motion recognition program using a captured image and a motion recognition program using the stored image stored in a medium to execute the method of any one of claims 1 to 10.

Images