KR102396639B1 - Method for playing images using photo card - Google Patents

Abstract

Disclosed is an image display system capable of displaying an image related to a photo image in augmented reality (AR) when a communication terminal is brought to a photo enhancement card after two photo cards are arranged side by side. According to the present invention, the image display system comprises: a communication terminal capturing two photo images from two photo cards while moving on the surfaces of two photo cards placed side by side at regular intervals to generate information about each of the captured images; and a management server searching for a plurality of pieces of content information matching first feature point information by using first feature point information extracted from each of captured image information and transmitting the plurality of searched content information to the communication terminal.

Description

{Method for playing images using photo card}

The present invention relates to an image reproduction technology, and more particularly, an image in which an image related to a photo image is displayed in augmented reality on a single screen when a communication terminal is brought to the photo augmentation card after arranging two photo cards side by side It relates to a regeneration system and method.

Functions of communication terminals are diversifying. For example, there are functions for data and voice communication, photo and video shooting through a camera, voice recording, music file playback through a speaker system, and output of images or videos to the display unit.

Some terminals add an electronic game play function or perform a multimedia player function. In particular, recent communication terminals can receive multicast signals that provide visual content such as broadcast and video or television programs.

As such terminals are diversified in functions, for example, in the form of a multimedia device (Multimedia player) equipped with complex functions such as taking pictures or videos, playing music or video files, playing games, and receiving broadcasts. is being implemented.

In order to support and increase the function of the terminal, it may be considered to improve the structural part and/or the software part of the terminal. An example of this is augmented reality (AR) technology.

Augmented reality is a graphic technique that shows virtual objects by synthesizing the real environment that the user sees with their eyes.

Therefore, since the research on the augmented reality technology started, research to apply the augmented reality in various fields has continued. Recently, due to the development of smart phones, various applications using augmented reality have been developed, and with the improvement of computing power of mobile devices such as communication terminals, PDA (Personal Digital Assistants), UMPS (Ultra Mobile Personal Computer) and the development of wireless network devices, Augmented reality systems in the hand have become possible.

An example of an application using such augmented reality is a service that provides information about buildings in a surrounding area (location) to a user using location information. In addition, there are services such as image search and location information service that provide search, mobile, shopping, advertisement, and object book by using markers or QR codes.

However, in the conventional augmented reality, only augmented reality for preset markers or objects is possible, but augmented reality cannot be realized by using complex patterns, gestures, and voices in combination.

In addition, there is a problem that the movement of the communication terminal is not reflected because it is at a level that only reproduces images already stored in the communication terminal.

In addition, even if two images are reproduced, there is a disadvantage in that it is difficult to distinguish the images in the images as they are reproduced in the same size. For example, as an image with a large number of people and an image with a few or one person are reproduced at the same size, it becomes difficult to distinguish between several people in an image with a large number of people.

1. Korean Patent No. 10-2028139 (Registration Date: September 26, 2019)

The present invention has been proposed to solve the problem according to the above background art, and after arranging two photo cards side by side, when the communication terminal is brought to the photo augmentation card, the image related to the photo image can be displayed in augmented reality. It is an object to provide a regeneration system and method.

In addition, in the present invention, after arranging two photo cards side by side, when the communication terminal is moved horizontally from the top of the photo augmentation card, the operation time of the image related to the photo image is synchronized using the photo card displayed in augmented reality. Another object of the present invention is to provide an image reproducing system and method for synchronizing an operation time of an image.

In addition, in the present invention, after arranging two photo cards side by side, when the communication terminal is moved horizontally from the top of the photo augmentation card, the operation time of the image related to the photo image is synchronized using the photo card displayed in augmented reality. Another object of the present invention is to provide an image reproducing system and method for synchronizing an operation time of an image.

Another object of the present invention is to provide an image reproducing system and method in which an image related to a photo image can be displayed in augmented reality by reflecting the movement of a communication terminal.

The present invention provides an image reproduction method in which an image related to a photo image can be displayed in augmented reality by arranging two photo cards side by side and then bringing the communication terminal to the photo augmentation card in order to achieve the above object.

The video playback method is

(a) generating respective photographed image information by photographing two photo images from the two photo cards as the communication terminal moves on the surfaces of two photo cards placed side by side at a predetermined interval;

(b) searching, by the management server, a plurality of pieces of content information matching the first feature point information using first feature point information extracted from each of the photographed image information;

(c) transmitting, by the management server, a plurality of pieces of searched content information to the communication terminal;

(d) displaying, by the communication terminal, a plurality of pieces of content information in parallel on a display screen; and

and (e) displaying, by the communication terminal, a plurality of contents information vertically on the display screen as the communication terminal recognizes the rotation of the communication terminal by 90° using a gyro sensor provided therein.

At this time, the horizontal width during the vertical display is changed to a certain ratio of the vertical width of the display screen 300 during the parallel display, and the vertical width during the vertical display is changed according to a preset aspect ratio. there is.

On the other hand, in another embodiment of the present invention, (a) the communication terminal detects the first of the two photo cards after the first movement time as it horizontally moves on the surface of two photo cards placed side by side at regular intervals generating first photographed image information by photographing one of the two photo images; (b) searching for one of two pieces of content information matching the first feature point information using first feature point information extracted from the first captured image information; (c) transmitting, by the management server, one of the found two pieces of content information to the communication terminal; (d) preparing, by the communication terminal, to output one of the two pieces of content information after a preset delay time in half of the display screen; (e) after the second movement time of the communication terminal, the second characteristic point information extracted from the second photographed image generated by photographing the remainder of the two photo cards according to whether the remaining two and outputting the remaining two pieces of the content information to the other half of the display screen after a predetermined time period to simultaneously reproduce the two pieces of content information. can do.

On the other hand, another embodiment of the present invention, (a) as the communication terminal moves on the surface of the two photo cards placed side by side at regular intervals, the two photo cards respectively arranged on the two photo cards generating two pieces of photographed image information corresponding to each by photographing an image; (b) searching, by the management server, two pieces of content information matching each of the two pieces of feature point information using two pieces of feature point information extracted from each of the two pieces of captured image information; (c) transmitting, by the management server, two pieces of searched content information to the communication terminal; and (d) the communication terminal compares the first face size of the first specific person and the second face size of the second specific person respectively displayed in the two pieces of the content information with each other, and according to the comparison result, two pieces of the contents It is possible to provide an image reproducing method using a photo card, comprising: enlarging any one or both of the information and outputting it on a display screen.

In this case, the step (d) includes: (d-1) displaying, by the communication terminal, two pieces of the content information on the display screen in a split screen; (d-2) determining, by the communication terminal, whether the first specific person and the second specific person are in specific positions having a predetermined width left and right from the center line of the divided screen on the divided screen, respectively; (d-3) according to the determination result, the communication terminal uses the first face size and the second focus area for the first specific person and the second focusing area for the second specific object, respectively generating a face size; (d-4) comparing, by the communication terminal, the first face size and the second face size; and (d-5) outputting, by the communication terminal, on the display screen by enlarging any one of the two pieces of content information according to the comparison result.

According to the present invention, after arranging two photo cards side by side, when the communication terminal is brought to the photo augmentation card, an image related to the photo image is displayed in augmented reality on one screen.

In addition, as another effect of the present invention, after arranging two photo cards side by side, if the communication terminal is moved horizontally from the top of the photo augmentation card, the operation time of the image related to the photo image is synchronized and displayed in augmented reality. can be heard

In addition, as another effect of the present invention, it is possible to display the image related to the photo image in augmented reality by reflecting the movement of the communication terminal.

In addition, as another effect of the present invention, after arranging two photo cards side by side, if the communication terminal is moved above the surface of the photo augmentation card, the size of the object in the video related to the photo image is enlarged by focusing, and augmented reality is displayed on the communication terminal. It can be said that it is displayed as

In addition, as another effect of the present invention, the image can be displayed in augmented reality by comparing the sizes of objects in two images to proportionally enlarge an image having a small object size to an image having a large object size.

1 is a block diagram of an image reproducing system using a photo card according to an embodiment of the present invention.
FIG. 2 is a block diagram of the communication terminal shown in FIG. 1 .
3 is an example of a horizontal screen according to an embodiment of the present invention.
4 is an example of a vertical screen reflecting the rotation of the communication terminal on the horizontal screen of FIG. 3 .
5 is a flowchart illustrating a process of extracting feature points from a photo image according to an embodiment of the present invention.
FIG. 6 is a conceptual diagram illustrating a structure of a database in which feature points extracted from a photo image and content information are mapped in the management server shown in FIG. 1 .
7 is a flowchart illustrating a process of generating a playback image by arranging it in a background according to an embodiment of the present invention.
8 is a flowchart illustrating a process of sensing a photo image of a photo card and receiving matching content as the communication terminal moves in the horizontal direction according to another embodiment of the present invention.
9 is a flowchart illustrating a process of sequentially displaying two contents on a screen after the sequence shown in FIG. 8 .
10 is an example of a screen when a communication terminal moves from left to right according to another embodiment of the present invention.
11 is an example of a screen when a communication terminal moves from right to left according to another embodiment of the present invention.
12 is a flowchart illustrating a process of sensing a photo image of a photo card and receiving matching content as the communication terminal moves in the horizontal direction according to another embodiment of the present invention.
13 is a flowchart illustrating a process of adjusting and displaying the size of an image by comparing object sizes in two pieces of content after the sequence shown in FIG. 12 .
14 is an example of a screen illustrating the concept of focusing the size of an object in two images while a reproduced image of the same size is displayed after the communication terminal moves according to another embodiment of the present invention.
FIG. 15 is an example of a screen that enlarges and displays a reproduced image having a relatively small object size after focusing shown in FIG. 14 .

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

In describing each figure, like reference numerals are used for like elements. Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The term “and/or” includes a combination of a plurality of related listed items or any of a plurality of related listed items.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. shouldn't

Hereinafter, an image reproducing system and method using a photo card according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an image reproducing system 1 using a photo card according to an embodiment of the present invention. Referring to FIG. 1 , a communication terminal 20 that captures photo images of photo cards 11 and 12 and extracts key point information, and a management server 40 that matches key point information with content information and transmits it to the communication terminal 20 ), the communication terminal 20 and the communication network 30 for connecting the management server 40 may be configured to include.

The photo cards 11 and 12 may include a first photo card 11 and a second photo card 12 . The first and second photo cards 11 and 12 have photo images 11-1 and 12-1 formed on their surfaces. In other words, the photo images 11-1 and 12-1 may be pictures of entertainers, pictures of performances, etc. as photos are attached to the surface of plastic, cardboard, or the like. Accordingly, the first photo image 11 - 1 and the second photo image 12 - 1 have different images.

The communication terminal 20 performs a function of extracting feature point information by photographing the photo images 11-1 and 12-1 of the photo cards 11 and 12. In addition, the communication terminal 20 matches the feature point information from the management server 40 , and performs a function of receiving and displaying content information related to the photo card.

The management server 40 performs a function of searching for content information using the feature point information received from the communication terminal 20 and providing it to the communication terminal 20 . Of course, the content information is prepared in advance and stored in the database 41 . The content information may include moving pictures, still images, background screens, photo image commentary information, and the like. In the case of a celebrity, the photo image commentary information may include a debut date, album list, lyrics, and the like.

The communication network 30 performs a function of connecting the communication terminal 20 and the management server 40 by communication. To this end, the communication network 30 refers to a connection structure capable of exchanging information between each node such as a plurality of terminals and servers, and includes a public switched telephone network (PSTN), a public switched data network (PSDN), and a comprehensive information network. (ISDN: Integrated Services Digital Networks), Broadband ISDN (BISDN), Local Area Network (LAN), Metropolitan Area Network (MAN: Metropolitan Area Network), Wide LAN (WLAN), etc. However, the present invention is not limited thereto, and wireless communication networks such as CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), Wibro (Wireless Broadband), WiFi (Wireless Fidelity), HSDPA ( It may be a High Speed Downlink Packet Access) network, a Bluetooth (bluetooth) network, a Near Field Communication (NFC) network, a satellite broadcasting network, an analog broadcasting network, a Digital Multimedia Broadcasting (DMB) network, and the like. Alternatively, it may be a combination of these wired communication networks and wireless communication networks.

Although it has been described in FIG. 1 that extracting the feature point is performed by the communication terminal 20, the communication terminal 20 transmits only the photographed image information to the management server 40, and extracting the feature point from the management server 40 It is also possible This is only a difference depending on where the program for extracting feature points is installed.

The management server 40 may be configured to include a microprocessor, a communication modem, a memory, and the like. The memory is a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (eg, SD (Secure Digital) or XD (eXtreme Digital)). memory, etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory , a magnetic disk, and an optical disk may include at least one type of storage medium. In addition, it may operate in relation to a web storage that performs a storage function on the Internet and a cloud server.

The database 41 may be configured in the management server 40, or alternatively, it may be configured in a separate database server.

FIG. 2 is a block diagram of the communication terminal 20 shown in FIG. 1 . Referring to FIG. 2 , the communication terminal 20 may be a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, or the like. can

The communication terminal 20 includes a wireless communication unit 110 , an A/V (Audio/Video) input unit 120 , a user input unit 130 , a sensing unit 140 , an output unit 150 , a memory 160 , an interface. It may include a unit 170 , a control unit 180 , and a power supply unit 190 . When these components are implemented in actual applications, two or more components may be combined into one component, or one component may be subdivided into two or more components as needed.

The wireless communication unit 110 may include a broadcast reception module 111 , a mobile communication module 112 , a wireless Internet module 113 , a short-range communication module 114 , and a GPS module 115 .

The broadcast receiving module 111 receives at least one of a broadcast signal and broadcast related information from an external broadcast management server (not shown) through a broadcast channel. In this case, the broadcast channel may include a satellite channel, a terrestrial channel, and the like. The broadcast management server may refer to a server that generates and transmits at least one of a broadcast signal and broadcast-related information, or a server that receives at least one of a previously generated broadcast signal and/or broadcast-related information and transmits it to a terminal.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, as well as a TV broadcast signal or a broadcast signal in which a data broadcast signal is combined with a radio broadcast signal. Broadcast-related information may be provided through a mobile communication network, and in this case, may be received by the mobile communication module 112 . Broadcast-related information may exist in various forms. For example, it may exist in the form of an Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or an Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives a broadcast signal using various broadcast systems. In particular, Digital Multimedia Broadcasting-Terrestrial (DMB-T), Digital Multimedia Broadcasting-Satellite (DMB-S), Media Forward Link Only (MediaFLO) ), Digital Video Broadcast-Handheld (DVB-H), Integrated Services Digital Broadcast-Terrestrial (ISDB-T), and the like can be used to receive digital broadcast signals. Also, the broadcast reception module 111 may be configured to be suitable for all broadcast systems that provide broadcast signals as well as such digital broadcast systems. A broadcast signal and/or broadcast-related information received through the broadcast reception module 111 may be stored in the memory 160 .

The mobile communication module 112 transmits/receives wireless signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include a voice call signal, a video call call signal, or various types of data according to text/multimedia message transmission/reception.

The wireless Internet module 113 refers to a module for wireless Internet access, and the wireless Internet module 113 may be built-in or external to the communication terminal 20 . The short-range communication module 114 refers to a module for short-range communication. As short-distance communication technologies, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, etc. may be used. The Global Position System (GPS) module 119 receives location information from a plurality of GPS satellites.

The A/V (Audio/Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 , a microphone 122 , and the like. The camera 121 processes an image frame such as a still image or a moving image obtained by an image sensor in a video call mode or a shooting mode. Then, the processed image frame may be displayed on the display unit 151 .

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110 . Two or more cameras 121 may be provided according to the configuration of the terminal. In addition, the positions may also be provided at regular intervals on the top and bottom and/or left and right.

The user input unit 130 generates key input data input by the user to control the operation of the terminal. The user input unit 130 may include a key pad, a dome switch, a touch pad (static pressure/capacitance), a jog wheel, a jog switch, a finger mouse, and the like. In particular, when the touch pad forms a layer structure with the display unit 151 to be described later, it may be referred to as a touch screen.

The sensing unit 140 generates a sensing signal for controlling the operation of the communication terminal 20 by detecting the current state of the communication terminal 20 such as the open/closed state, location, user contact, etc. of the communication terminal 20 . For example, when the communication terminal 20 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. In addition, it may be responsible for sensing functions related to whether the power supply unit 190 supplies power, whether the interface unit 170 is coupled to an external device, and the like.

Also, the sensing unit 140 may include a proximity sensor 141 . The proximity sensor 141 can detect an approaching object or the presence or absence of an object existing in the vicinity without mechanical contact. The proximity sensor 141 may detect a proximity object by using a change in an alternating current or static magnetic field, or a rate of change in capacitance, or the like. Proximity sensor 141 may be provided with two or more depending on the configuration aspect.

Also, the sensing unit 140 may include a gyro sensor 142 . The gyro sensor 142 is a concept including a sensor for detecting the movement of an object using a gyroscope, an inertial sensor, and an acceleration sensor. Gyroscopes include mechanical gyroscopes, ring laser gyroscopes, and fiber optic gyroscopes. The gyro sensor 142 provides a signal for controlling the communication terminal by detecting the movement of the communication terminal.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal, and includes a display unit 151 , a sound output module 153 , an alarm unit 155 , a vibration module 157 , and the like. may be included.

The display unit 151 displays and outputs information processed by the communication terminal 20 . For example, when the communication terminal 20 is in a call mode, a user interface (UI) or graphic user interface (GUI) related to a call is displayed. In addition, when the communication terminal 20 is in the video call mode or the shooting mode, the captured or received images may be displayed individually or simultaneously, and a UI and a GUI may be displayed.

In addition, the display unit 151 is a liquid crystal display (liquid crystal display), a thin film transistor-liquid crystal display (thin film transistor-liquid crystal display), an organic light-emitting diode (organic light-emitting diode), a flexible display (flexible display), three-dimensional At least one of a display (3D display) may be included. In addition, two or more display units 151 may exist according to an implementation form of the communication terminal 20 . For example, the communication terminal 20 may be provided with an external display unit (not shown) and an internal display unit (not shown) at the same time.

The sound output module 153 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. In addition, the sound output module 153 outputs a sound signal related to a function performed in the communication terminal 20 , for example, a call signal reception sound, a message reception sound, and the like. The sound output module 153 may include a speaker, a buzzer, and the like.

The alarm unit 155 outputs a signal for notifying the occurrence of an event in the communication terminal 20 . Examples of events occurring in the communication terminal 20 include call signal reception, message reception, and key signal input. The alarm unit 155 outputs a signal for notifying the occurrence of an event in a form other than an audio signal or a video signal. For example, the signal may be output in the form of vibration.

When a call signal is received or a message is received, the alarm unit 155 may output a signal for notifying it. Also, when a key signal is input, the alarm unit 155 may output a signal as feedback to the key signal input. The user may recognize the occurrence of an event through a signal output from the alarm unit 155 . Of course, a signal for notifying of the occurrence of an event may also be output through the display unit 151 or the sound output module 153 . The alarm unit 155 may configure a software component for this purpose.

The vibration module 157 may generate vibrations of various intensities and patterns according to the vibration signal transmitted by the controller 180 . The intensity, pattern, frequency, movement direction, movement speed, etc. of the vibration generated by the vibration module 157 can be set by the vibration signal, and two or more vibration modules 157 may be provided according to the configuration aspect. The vibration module 157 generates vibration in the communication terminal 20 and may include a vibration motor or a vibration element.

The memory 160 may store a program for processing and/or control of the controller 180, and is for temporary storage of input or output data (eg, phonebook, message, still image, video, etc.). function can also be performed. In addition, a program, software, data, etc. having an algorithm for extracting feature points by processing a photo image captured by the camera 121 may be stored.

The memory 160 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory, etc.), RAM , and may include a storage medium of at least one type of ROM. Also, the communication terminal 20 may operate a web storage that performs a storage function of the memory 150 on the Internet.

The interface unit 170 functions as an interface with all external devices connected to the communication terminal 20 . Examples of external devices connected to the communication terminal 20 include a wired/wireless headset, an external charger, a wired/wireless data port, a memory card, a card socket such as a SIM/UIM card, and an audio I/O (Input). /Output) terminal, video I/O (Input/Output) terminal, earphone, etc. The interface unit 170 may receive data or receive power from such an external device and transmit it to each component inside the communication terminal 20, and may allow the data inside the communication terminal 20 to be transmitted to an external device. .

The controller 180 generally controls the overall operation of the communication terminal 20 by controlling the operation of each unit. For example, it performs related control and processing for voice calls, data communications, video calls, and the like. Also, the controller 180 may include a multimedia playback module 181 for multimedia playback. The multimedia playback module 181 may be configured as hardware in the control unit 180 or may be configured as software separately from the control unit 180 . Also, the controller 180 may perform a function of executing an algorithm for extracting feature points by processing the photographed photo image.

The power supply unit 190 receives external power and internal power under the control of the control unit 180 to supply power required for operation of each component.

3 is an example of a horizontal screen according to an embodiment of the present invention. Referring to FIG. 3 , it is assumed that the first photo card 11 is disposed on the right side and the second photo card 12 is disposed on the left side of the first photo card 11 at regular intervals. In this arrangement, when the communication terminal 20 is moved from the right to the left at a predetermined height over the surfaces of the first photo card 11 and the second photo card 120, the display screen 300 of the communication terminal 20 is The first reproduced image 311 and the second reproduced image 312 are sequentially or simultaneously displayed on the screen. The first playback image 311 and the second playback image 312 are the first and second photo images 11-1 and 12 configured in the first photo card 11 and the second photo card 120 as content information. It is information related to -1). That is, the first reproduced image 311 is displayed and reproduced first, and then the second reproduced image 312 is displayed and reproduced. Of course, it is also possible that the first reproduced image 311 is first displayed and not reproduced, and after a black screen for a certain period of time, it is reproduced simultaneously with the second reproduced image 311 .

On the other hand, when the communication terminal 20 is moved from the left to the right, the first reproduced image 311 is displayed and played on the display screen 300 of the communication terminal 20, and then the second reproduced image 312 is displayed. becomes and plays That is, the positions of the first reproduced image 311 and the second reproduced image 312 shown in FIG. 3 are opposite to each other.

In FIG. 3 , it is assumed that the camera 121 is installed in the communication terminal 20 in the longitudinal direction, but when two cameras (121 in FIG. 2 ) are installed at regular intervals on the left and right sides of the communication terminal 20, top and bottom It is possible to move the communication terminal 20 in an upward direction under or under the direction. In this case, the first reproduced image 311 and the second reproduced image 312 may be simultaneously displayed and reproduced.

In addition, it is characterized in that the background 301 displayed on the display screen 300 of the communication terminal 20 is the same background image as the surface of the object on which the photo cards 11 and 12 are placed. For example, when the photo cards 11 and 12 are placed on the surface of a table having a wood pattern, the background 301 becomes a wood pattern image. That is, the communication terminal 20 captures the photo cards 11 and 12 and the background on which the photo cards 11 and 12 are placed at the same time, and separates the photo cards 11 and 12 from the photo card 11, 12) extracts the image extracted from the feature points, and extracts the background image from the background.

FIG. 4 is an example of a vertical screen reflecting the rotation of the communication terminal 20 on the horizontal screen of FIG. 3 . Referring to FIG. 4 , when the user rotates the communication terminal 20 by 90° using the gyro sensor ( 142 in FIG. 2 ) configured in the communication terminal 20 , the first playback image up and down on the display screen 300 . 311 and a second playback image 312 are displayed.

Of course, in this case, the reproduction sizes of the first reproduced image 311 and the second reproduced image 312 are changed to fit the vertical width of the display screen 300 . For example, assuming that the horizontal width of the display screen 300 is 15 cm and the vertical width is 7 cm, the width of the reproduced image 311 in FIG. 3 may be about 6 cm and the vertical width may be about 4 cm. On the contrary, in the case of FIG. 4 , when 90% of the vertical width of the display screen 300 is converted into the horizontal width of the reproduced screen size of the reproduced images 311 and 312, it is about 6.3 cm. That is, only 90% is used considering the empty space on both sides. Of course, values between 90% and 95% are also possible. In addition, if the aspect ratio of the reproduction screen size of the reproduction images 311 and 312 is applied to the 4:3 ratio, the vertical width may be about 4.7 cm.

5 is a flowchart illustrating a process of extracting feature points from photo images 11-1 and 12-1 according to an embodiment of the present invention. In general, in the image processing field, in order to recognize and process an input image, a feature point is extracted from the input image, a descriptor is created through the extracted feature point, and object recognition and tracking are performed through matching with other images.

First, an integral image having a brightness value is generated with respect to photographed image information generated by photographing the photo images 11-1 and 12-1 through an integral image generator (not shown) (step S510).

Thereafter, the feature point detector (not shown) generates a feature point candidate group that can become a feature point in the captured image information by using the generated integral image for each filter size, and extracts the feature points through mutual comparison with the generated feature point candidate group. becomes (step S520). Here, the feature point extraction may be performed using a Harris corner extraction technique, a Scale-Invariant Feature Transform (SIFT), a Speeded UP Robust Features (SURF), or an Oriented BRIEF (ORB) algorithm.

Thereafter, the keypoint generated in this way has an image around the keypoint through a descriptor generator (not shown) and the main direction of the keypoint is set (step S530).

Thereafter, the main direction generated for each feature point is rotated in one direction and fixed, and the information is stored as a descriptor and generated as a database (step S540). The descriptor includes matching information such as brightness, color, and gradient direction of the detected feature point.

Thereafter, using this descriptor, a descriptor that is stored and stored in a database in advance is matched (step S550). The matching method may be a method of finding the same descriptor by examining them one by one, or using the number of matching points.

Accordingly, content information related to the photo images 11 - 1 and 12 - 1 is searched and transmitted to the communication terminal 20 . Since an algorithm for detecting a feature point from such an actual image is well known, a further description will be omitted for a clear understanding of the present invention. In addition, the image recognition algorithm described with reference to FIG. 5 is merely an example, and other image recognition algorithms may be used.

On the other hand, the integral image generator (not shown), the feature point detector (not shown), the descriptor generator (not shown), etc. mean a unit that processes at least one function or operation, which may be implemented in software and/or hardware. . In hardware implementation, application specific integrated circuit (ASIC), digital signal processing (DSP), programmable logic device (PLD), field programmable gate array (FPGA), processor, microprocessor, other It may be implemented as an electronic unit or a combination thereof.

In software implementation, software composition component (element), object-oriented software composition component, class composition component and task composition component, process, function, attribute, procedure, subroutine, segment of program code, driver, firmware, microcode, data , databases, data structures, tables, arrays, and variables. Software, data, etc. may be stored in a memory and executed by a processor. The memory or processor may employ various means well known to those skilled in the art.

FIG. 6 is a conceptual diagram illustrating a structure of a database (41 of FIG. 1 ) that maps feature points extracted from a photo image and content information (eg, a reproduced image) in the management server shown in FIG. 1 . Referring to FIG. 6 , feature points and content information are mapped and stored in the database 41 in advance.

In other words, the management server 40 extracts a feature point from content information related to the photo image in advance, and matches the feature point with the content information based on the extracted feature point. For example, feature points are extracted from image frames (eg, image 1 is exemplified), and content information is matched for each feature point.

For example, as shown in FIG. 6 , the feature point 1_1, the feature point 1_2, the feature point 1_3 belonging to the feature point 1 ... AR (Augmented Reality) content 1_1, AR content 1_2, AR content 1_3 belonging to augmented reality content 1 in … this can be That is, key point 1-1 is mapped to AR content 1-1, key point 1-2 is mapped to AR content 1-2, and key point 1-3 is mapped to AR content 1-3.

Of course, this key point extraction process is performed through steps S510 to S540 shown in FIG. 5 . In other words, an integral image is generated from image frames constituting the content information, and a feature point is extracted from the integral image.

In addition, when the feature points extracted from the photo images 11-1 and 12-1 correspond to 1_3, all AR contents corresponding to the downward direction from the AR contents 1_3 are transmitted to the communication terminal 20, and the communication terminal 20 plays the corresponding contents sequentially. Of course, when the feature points extracted from the photo images 11-1 and 12-1 correspond to 1_1, the AR contents 1_1, 1_2, and 1_3 corresponding to the downward direction from the AR contents 1_1 are all transmitted to the communication terminal 20, The communication terminal 20 sequentially reproduces the corresponding contents.

7 is a flowchart illustrating a process of arranging and generating a playback image (content image) in a background according to an embodiment of the present invention. Referring to FIG. 7 , images of target objects (ie, photo images 11-1 and 12-1) are captured (step S710). The target objects are positioned on the surface of a predefined object (eg, a desk, a floor, a chair, a calendar, a picture, etc.). Therefore, it is captured as a background image. Target objects may include, but are not limited to, 1-D, 2-D, and 3-D objects.

Thereafter, the image may be segmented into regions corresponding to target objects and regions corresponding to a predefined background ( 301 in FIG. 3 ). Segmenting the image may include separating a predefined background from the image (step S720).

Thereafter, regions corresponding to the target objects are converted into digital information (step S730).

Thereafter, the AR objects (ie, content information) are matched and displayed on the background 301 of the display screen 300 (step S740). AR objects may correspond to target objects, and may be generated based at least in part on a digital image. AR objects are generated based on detection of two-dimensional (2-D) blobs (binary large objects) within a captured image, wherein detection of 2-D blobs is one or more target objects positioned on a predefined background. may be based on the surface reflectance associated with them.

AR objects may be edited in response to user input. Editing may include dividing at least one of the AR objects into a plurality of AR portions. The user input may designate a predetermined AR portion from a plurality of AR portions. Then, the designated AR portion may be manipulated based on the user manipulation command input signal.

Manipulating the AR portion designated on the basis of the user manipulation command input signal is to add depth to the AR portion designated based on the determined finger pressure on the display screen 300 of the communication terminal 20, Rotating the designated AR part based on the user's gesture performed on the display screen 300, assembling the designated AR part into one or more layers, and displaying the designated AR part on the display screen 300 of the communication terminal 20 moving to one or more locations displayable on the image, or projecting a designated AR portion, and the like.

8 is a flowchart illustrating a process of sensing a photo image of a photo card and receiving matching content as the communication terminal 20 moves in a horizontal direction according to another embodiment of the present invention. Referring to FIG. 8 , it is assumed that the first photo card 11 is arranged on the left side and the second photo card 12 is arranged on the right side of the first photo card 11 at regular intervals.

In this arrangement, when the communication terminal 20 is horizontally moved from left to right (arrow direction) at a predetermined height above the surfaces of the first photo card 11 and the second photo card 120, the communication terminal 20 is A first playback image 311 is displayed on the display screen 300 . In other words, as the camera 121 installed on the right side of the communication terminal 20 senses the first photo image 11-1 of the first photo card 11, the first photo image 11-1 related to the first photo image 11-1 is 1 The reproduced image 311 is output to half (ie, the left half) of the display screen 300 .

The output may be made after a certain period of time has elapsed. For example, about 3 to 5 seconds after the camera 121 senses the first photo image 11-1, the first reproduced image 311 may start to be output.

Of course, the second playback image (not shown) is not output until the camera 121 senses the second photo image 12 - 1 of the second photo card 12 .

The first playback image 311 and the second playback image are the first and second photo images 11-1 and 12-1 configured in the first photo card 11 and the second photo card 12 as content information. is information related to

After the communication terminal 20 senses the first photo card 11, if it continues to move to the right, the second photo image 12-1 of the second photo card 12 is sensed, and a second reproduction related thereto is sensed. An image (not shown) is output to the other half of the display screen 300 .

In addition, the background 301 displayed on the display screen 300 of the communication terminal 20 may be the same background image as the surface of the object on which the photo cards 11 and 12 are placed. For example, when the photo cards 11 and 12 are placed on the surface of a table having a wood pattern, the background 301 becomes a wood pattern image. That is, the communication terminal 20 captures the photo cards 11 and 12 and the background on which the photo cards 11 and 12 are placed at the same time, and separates the photo cards 11 and 12 from the photo card 11, 12) extracts the image extracted from the feature points, and extracts the background image from the background.

9 is an example of a screen when the communication terminal 20 horizontally moves from right to left according to an embodiment of the present invention. In FIG. 9, unlike FIG. 8, the communication terminal 20 moves in the horizontal direction from right to left on the surfaces of the first and second photo cards 11 and 12 arranged side by side. Accordingly, unlike FIG. 3 , in FIG. 4 , the communication terminal 20 senses the second photo card 12 first.

When the communication terminal 20 is moved from the right to the left, the second reproduced image 312 is displayed on the display screen 300 of the communication terminal 20 , and then the first reproduced image 311 is displayed. That is, the positions of the first reproduced image 311 and the second reproduced image 312 shown in FIG. 8 are opposite to each other.

In other words, as the camera 121 installed on the left side of the communication terminal 20 senses the second photo image 12-1 of the second photo card 12, the second photo image 12-1 is related to the second photo image 12-1. 2 The reproduced image 312 is output to a half (ie, a right half) of the display screen 300 .

The output may be made after a certain period of time has elapsed. For example, after about 1 to 2 seconds after the camera 121 senses the second photo image 12 - 1 , the second playback image 312 may start to be output.

In other words, in the case of FIG. 8 , the first reproduced image 311 and the second reproduced image 312 may be simultaneously output. To this end, the output time of the first reproduced image 311 is set longer than the output time of the second reproduced image 312 . Of course, in the case of FIG. 3 , the output time of the second reproduced image 312 is set longer than the output time of the first reproduced image 311 . Accordingly, it is possible to synchronize (synchronize) the operation time of the reproduced image.

Of course, it is also possible to output the operation time points of the first reproduced image 311 and the second reproduced image 312 in the order they are sensed, without synchronizing them. In other words, any one of the first reproduced image 311 and the second reproduced image 312 may be output first. For example, the first reproduced image 311 may be output first, and the second reproduced image 312 may be sequentially output later. Of course, according to the moving direction of the communication terminal 20 , the second reproduced image 312 may be output first, and the first reproduced image 311 may be sequentially output later.

In addition, a black still image may be output for a predetermined period of time before the operation time points of the first reproduced image 311 and the second reproduced image 312 are synchronized. In other words, a black still image is output before the first playback image 311 is output, and then the first playback image 311 and the second playback image 312 are sequentially output. Of course, the black still image is output on the half of the display screen 300 on which the first playback image 311 is output.

For example, assuming that the width of the display screen 300 is 15 cm and the vertical width is 7 cm in a state in which the communication terminal 20 is laid down in the horizontal direction, half of the display screen 300 has a width of 7.5 cm, and the vertical width is the same as 7 cm. In this case, the size of the reproduced images 311 and 312 may be 7 cm in width and 5 cm in vertical width.

In FIG. 9 , it is assumed that the camera 121 is installed in the communication terminal 20 in the longitudinal direction, but when two cameras ( 121 in FIG. 2 ) are installed at regular intervals on the left and right sides of the communication terminal 20 , top and bottom It is possible to move the communication terminal 20 in an upward direction under or under the direction. In this case, the first reproduced image 311 and the second reproduced image 312 may be simultaneously displayed and reproduced.

10 is an example of a screen when a communication terminal moves from left to right according to another embodiment of the present invention. Referring to FIG. 10 , it is assumed that the first photo card 11 is disposed on the left side and the second photo card 12 is disposed on the right side of the first photo card 11 at regular intervals.

In this arrangement, when the communication terminal 20 is horizontally moved from left to right (arrow direction) at a predetermined height above the surfaces of the first photo card 11 and the second photo card 120, the communication terminal 20 is A first playback image 311 is displayed on the display screen 300 . In other words, as the camera 121 installed on the right side of the communication terminal 20 senses the first photo image 11-1 of the first photo card 11, the first photo image 11-1 related to the first photo image 11-1 is 1 The reproduced image 311 is output to half (ie, the left half) of the display screen 300 .

The first playback image 311 and the second playback image are the first and second photo images 11-1 and 12-1 configured in the first photo card 11 and the second photo card 12 as content information. is information related to

The output may be made after a certain period of time has elapsed. For example, about 3 to 5 seconds after the camera 121 senses the first photo image 11-1, the first reproduced image 311 may start to be output.

Of course, the second playback image (not shown) is not output until the camera 121 senses the second photo image 12 - 1 of the second photo card 12 . In other words, the first reproduced image 311 is in the output standby state until a predetermined first delay time. At this time, the screen is not visually visible to the user of the communication terminal 20 . In other words, since there may not be a person image from the first frame, the first playback image of the visible view so that the user can visually recognize it after the time taken from the frame with the person image to the frame with the movement of the person image (311) is output.

Also, it may include a person frame time having a time taken from a frame having a person image to an nth frame in which the degree of motion exceeds a reference level.

Also, the second playback image (not shown) has such a second delay time. That is, the screen is visually invisible to the user of the communication terminal 20 . Accordingly, the first reproduced image 311 and the second reproduced image are selected from the sum of the horizontal movement time and the second delay time during which the communication terminal 20 horizontally moves from the first photo card to the second photo card, and the first delay time. After waiting for a longer time, it is simultaneously output to be visually displayed to the user. Of course, the second delay time also includes a person frame time having a time taken from the frame in which the person image is located to the nth frame in which the movement of the person image occurs at a certain level or more.

After the communication terminal 20 senses the first photo card 11, if it continues to move to the right, the second photo image 12-1 of the second photo card 12 is sensed, and a second reproduction related thereto is sensed. An image (not shown) is output to the other half of the display screen 300 .

In addition, the background 301 displayed on the display screen 300 of the communication terminal 20 may be the same background image as the surface of the object on which the photo cards 11 and 12 are placed. For example, when the photo cards 11 and 12 are placed on the surface of a table having a wood pattern, the background 301 becomes a wood pattern image. That is, the communication terminal 20 captures the photo cards 11 and 12 and the background on which the photo cards 11 and 12 are placed at the same time, and separates the photo cards 11 and 12 from the photo card 11, 12) extracts the image extracted from the feature points, and extracts the background image from the background.

11 is an example of a screen when a communication terminal moves from right to left according to another embodiment of the present invention. In FIG. 11, unlike FIG. 10, the communication terminal 20 moves in the horizontal direction from right to left on the surfaces of the first and second photo cards 11 and 12 arranged side by side. Accordingly, unlike FIG. 10 , in FIG. 11 , the communication terminal 20 senses the second photo card 12 first.

When the communication terminal 20 is moved from the right to the left, the second reproduced image 312 is displayed on the display screen 300 of the communication terminal 20 , and then the first reproduced image 311 is displayed. That is, the positions of the first reproduced image 311 and the second reproduced image 312 shown in FIG. 3 are opposite to each other.

In other words, as the camera 121 installed on the left side of the communication terminal 20 senses the second photo image 12-1 of the second photo card 12, the second photo image 12-1 is related to the second photo image 12-1. 2 The reproduced image 312 is output to a half (ie, a right half) of the display screen 300 .

The output may be made after a certain period of time has elapsed. For example, after about 1 to 2 seconds after the camera 121 senses the second photo image 12 - 1 , the second playback image 312 may start to be output.

In other words, in the case of FIG. 10 , the first reproduced image 311 and the second reproduced image 312 may be simultaneously output. To this end, the output time of the first reproduced image 311 is set longer than the output time of the second reproduced image 312 . Of course, in the case of FIG. 3 , the output time of the second reproduced image 312 is set longer than the output time of the first reproduced image 311 . Accordingly, it is possible to synchronize (synchronize) the operation time of the reproduced image.

Of course, it is also possible to output the operation time points of the first reproduced image 311 and the second reproduced image 312 in the order they are sensed, without synchronizing them. In other words, any one of the first reproduced image 311 and the second reproduced image 312 may be output first. For example, the first reproduced image 311 may be output first, and the second reproduced image 312 may be sequentially output later. Of course, according to the moving direction of the communication terminal 20 , the second reproduced image 312 may be output first, and the first reproduced image 311 may be sequentially output later.

Also, before the operation time points of the first reproduced image 311 and the second reproduced image 312 are synchronized, it may be invisible on the display screen 300 for a predetermined period of time. In other words, it is in an invisible view state before the first reproduced image 311 is output, and then the first reproduced image 311 and the second reproduced image 312 are sequentially output. Of course, the invisible view is displayed on a half of the display screen 300 on which the first reproduced image 311 is to be output.

For example, assuming that the width of the display screen 300 is 15 cm and the vertical width is 7 cm in a state in which the communication terminal 20 is laid down in the horizontal direction, half of the display screen 300 has a width of 7.5 cm, and the vertical width is the same as 7 cm. In this case, the size of the reproduced images 311 and 312 may be 7 cm in width and 5 cm in vertical width.

In FIG. 10 , it is assumed that the camera 121 is installed in the communication terminal 20 in the longitudinal direction, but when two cameras ( 121 in FIG. 2 ) are installed at regular intervals on the left and right sides of the communication terminal 20 , the top and bottom It is possible to move the communication terminal 20 in an upward direction under or under the direction. In this case, the first reproduced image 311 and the second reproduced image 312 may be simultaneously displayed and reproduced.

12 is a flowchart illustrating a process of sensing a photo image of a photo card and receiving matching content as the communication terminal 20 moves in a horizontal direction according to another embodiment of the present invention. Referring to FIG. 8 , when the communication terminal 20 is moved horizontally on the surfaces of the first and second photo cards 11 and 12 arranged side by side, the image sensor configured in the camera 121 sequentially moves the first The photo image 11-1 of the photo card 11 and the photo image 12-1 of the second photo card 12 are sensed to generate photographed image information (steps S1210 and S1220).

Of course, if the movement direction of the communication terminal 20 is opposite, the photo image 12-1 of the second photo 120 and the photo image 11-1 of the first photo card 11 may be sensed sequentially. there is. Alternatively, it is also possible to move the communication terminal 20 in the vertical direction. In this case, the photo image 11 - 1 of the first photo card 11 and the photo image 12 - 1 of the second photo card 12 may be simultaneously sensed.

When the communication terminal 20 transmits the photographed image information to the management server 40, the management server 40 extracts photo image-related feature point information from the photographed image information (step S1230).

Thereafter, the management server 40 searches the database ( 41 of FIG. 1 ) for contents matching the feature point information and transmits it to the communication terminal 20 (step S1240 ).

Of course, step 1230 may also be executed in the communication terminal 20 . In other words, it is a method of configuring software having an algorithm for extracting key point information in the communication terminal 20 , extracting key point information using this software, and transmitting it to the management server 40 .

13 is a flowchart illustrating a process of adjusting and displaying the size of an image by comparing object sizes in two pieces of content after the sequence shown in FIG. 12 . Referring to FIG. 9 , the communication terminal 20 loads the contents transmitted by the management server 40 and outputs the first playback image 311 and the second playback image 312 on the display screen (300 in FIG. 3 ). do (step S1310).

The communication terminal 20 calculates the size of a face by detecting a person in front of the center of the screen (ie, at the forefront) of the first reproduced image 311 and the second reproduced image 312 by focusing (step S1320).

Thereafter, it is determined whether the calculated face sizes are the same (step S1330). That is, the sizes of the first focusing area 311 - 2 and the second focusing area 312 - 2 are compared with each other.

As a result of the comparison, if the sizes are the same in step S1330, the communication terminal 20 maintains the current aspect ratio (step S1331).

On the other hand, in step S1330, if the size is different, that is, if the face size of the person displayed in the first reproduced image 311 is larger than the face size of the person displayed in the second reproduced image 312, the first reproduced image 311 The screen of the second reproduced image 312 is enlarged based on the face size of the person displayed in (step S1340).

Of course, in steps S1330 to S1340, it is assumed that the face sizes of the specific persons 311-1 and 312-1 are all larger than a preset reference value.

Contrary to this, when the face sizes of the specific persons 311-1 and 312-1 are all smaller than a preset reference value, the two reproduced images 311 and 312 are enlarged and output at a preset ratio. Of course, the magnification ratio of the first playback image 311 and the magnification ratio of the second playback image 312 may be different from each other. For example, the magnification ratio of the first playback image 311 may be about 110%, and the magnification ratio of the second playback image 312 may be about 125%. Of course, the smaller the face size, the higher the magnification ratio.

14 is an example of a screen illustrating the concept of focusing the size of an object in two images while a reproduced image of the same size is displayed after the communication terminal moves according to another embodiment of the present invention.

Referring to FIG. 14 , it is assumed that the first photo card 11 is arranged on the right side and the second photo card 12 is arranged on the left side of the first photo card 11 at regular intervals.

In this arrangement state, when the communication terminal 20 is horizontally moved from the left to the right or from the right to the left at a predetermined height over the surfaces of the first photo card 11 and the second photo card 120, the communication terminal 20 The first reproduced image 311 and the second reproduced image 312 are displayed on the display screen 300 of .

Of course, it is also possible to move the communication terminal 20 in the vertical direction. In other words, on the surface of the first photo card 11 and the second photo card 120 that are arranged side by side at regular intervals, the first photo card 11 and the second photo card 120 are moved from the top to the bottom. can Of course, alternatively, the communication terminal 20 may be moved from the bottom to the top or from the top to the bottom of the first photo card 11 and the second photo card 120 .

After the communication terminal 20 moves, the first reproduced image 311 and the second reproduced image 312 are displayed on the display screen 300 of the communication terminal 20 . The first playback image 311 is content related to the first photo image 11-1, and the second playback image 312 is content related to the second photo image 12-1.

The controller 180 of the communication terminal 20 calculates a focusing area 311-2 of the first object 311-1 in the first reproduced image 311 using the installed face recognition algorithm program, and reproduces the second image. In the image 312 , a focusing area 312-2 of the second object 312-1 is calculated.

The first object 311-1 and the second object 312-1 may be a person, but are not limited thereto, and may be an animal, an object, a tree, or the like. In an embodiment of the present invention, a description will be made assuming a person.

The object recognition technology in the video, especially the face recognition technology, is widely known including algorithms using Python code, so further explanation will be omitted. In addition, techniques for improving the recognition rate in such face recognition have been proposed.

In other words, face recognition is a technology that recognizes a person by extracting features from a face image, learning it through various algorithms, and comparing the learned data with features in a new face image. Various methods are required to improve the recognition rate. It is a technology For face recognition, it is necessary to extract feature components from face images in the learning stage, and for this purpose, an existing facial feature component extraction method includes a Linear Discriminant Analysis (LDA) method. In this method, facial images are expressed as dots in a high-dimensional space, and features for identifying people are extracted by analyzing class information and the distribution of dots.

Of course, the face area is detected, and unnecessary areas are removed through the facial outline calculated using a Gabor filter in the detected face area, and then the size of the face area is normalized to a certain size, and the normalized face image is linearized. A method that can improve the recognition rate by about 13% compared to the existing face recognition method by extracting facial feature components through discriminant analysis and performing face recognition by learning through an artificial neural network is also proposed.

14 , the focusing areas 311 - 2 and 312 - 2 are areas made up of pixels constituting the face and are sized. In addition, the focusing areas 311 - 2 and 312 - 2 preferably have a circular shape, but may also have a rectangular shape. In other words, the focusing areas 311 - 2 and 312 - 2 are enlarged or reduced according to the size of the recognized face area. That is, when a large number of activated pixels in the recognized face region is recognized, the size of the focusing region increases, and when a small number of inactive pixels is recognized, the size of the focusing region decreases.

Accordingly, a difference in face size occurs according to the first focusing area 311 - 2 and the second focusing area 312 - 2 , and the radius of the distance from the center point to the boundary can be calculated because the shape is circular. In general, the unit of radius is cm, and the equation for converting pixels to cm is as follows.

Here, DIP (Device Independence Pixel) is the number of pixels per inch (=2.54 cm), and is a unit indicating pixels that fit within one inch. That is, DIP is a unit for outputting at the same ratio in any screen size.

Through this conversion process, sizes of the first focusing area 311 - 2 and the second focusing area 312 - 2 are calculated. For example, the radius r1 of the first focusing area 311 - 2 may be calculated as 1 cm, and the radius r2 of the second focusing area 312 - 2 may be calculated as 0.5 cm.

Of course, when the first focusing area 311 - 2 and the second focusing area 312 - 2 are in a rectangular shape, they can be compared with each other in terms of horizontal and vertical area sizes. For example, the area size of the first focusing area 311 - 2 may be calculated to be about 1 cm x 0.5 cm, and the area size of the second focusing area 312 - 2 may be calculated to be about 0.5 cm x 0.25 cm, respectively.

In addition, the first object (ie, a person) 311-1 and the second object 312-1 are located in the center of the screen of the first playback image 311 and the second playback image 312 or at a position close to the center. can be an object. In other words, assuming that the horizontal width of the display screen 300 is about 15 cm and the vertical width is about 7 cm in a state in which the communication terminal 20 is laid down in the horizontal direction, the display screen 300 based on the boundary line 303 ) half (that is, split screen) has a width of 7.5cm and a height of 7cm. In this case, the size of the reproduced images 311 and 312 may be 7 cm in width and 5 cm in vertical width. Therefore, the center of the screen is 3.5 cm, and there are cases where the person is not in the center, so it can be assumed that the person within the range of 3.25 cm to 3.75 cm is also in the center.

Of course, a person may exist outside the range of 3.25 cm to 3.75 cm. That is, there may be no candidate persons within this range. In this case, the person may be searched for by increasing the range to a predetermined width. For example, increasing the width to 3.15 cm to 3.85 cm is mentioned. Of course, if a person is not searched even when the width is increased, the content information 311 and 312 maintain the current state.

Also, there may be several (two or more) candidates within the range of 3.25 cm to 3.75 cm. In this case, the faces of all candidate persons are recognized, they are converted into face sizes through focusing, and the single smallest candidate person can be determined as a specific person to be compared by comparing them with each other.

Of course, as shown in FIG. 14 , when the face sizes of the people output to the first playback image 311 and the second playback image 312 are larger than a preset reference value, the first playback image 311 is If the face size of the person is larger than the face size of the person in the second reproduced image 312 , the screen size of the second reproduced image 312 is enlarged at a predetermined ratio. This principle can also be applied in the opposite case.

On the other hand, there may be a case where both the face sizes of the people output to the first reproduced image 311 and the second reproduced image 312 are larger than a preset reference value. In this case, if the face size of the person in the first reproduced image 311 is larger than the face size of the person in the second reproduced image 312, both the screens of the first reproduced image 311 and the second reproduced image 312 are displayed. It can be enlarged, and the enlargement ratio can be adjusted differently for each face size. Of course, this enlargement ratio may be preset and stored in the communication terminal 20 .

Of course, if the face size of the person in the first reproduced image 311 and the face size of the person in the second reproduced image 312 are the same, the screen is enlarged at the same rate.

In addition, the background 301 displayed on the display screen 300 of the communication terminal 20 may be the same background image as the surface of the object on which the photo cards 11 and 12 are placed. For example, when the photo cards 11 and 12 are placed on the surface of a table having a wood pattern, the background 301 becomes a wood pattern image. That is, the communication terminal 20 captures the photo cards 11 and 12 and the background on which the photo cards 11 and 12 are placed at the same time, and separates the photo cards 11 and 12 from the background to separate the photo cards 11, 12) extracts the image extracted from the feature points, and extracts the background image from the background.

FIG. 15 is an example of a screen for magnifying and displaying a reproduced image having a relatively small object size after focusing shown in FIG. 14 . Referring to FIG. 15 , according to a size comparison between the first focusing area 311 - 2 and the second focusing area 312 - 2 , the first focusing area 311 - 2 is larger than the second focusing area 312 - 2 . If it is determined to be large, the controller 180 of the communication terminal 20 enlarges the screen of the second reproduced image 312 corresponding to the second focusing area 312-2 belonging to the smaller side and outputs it on the display screen 301 . .

For example, before the screen enlargement, assuming that the horizontal width of the display screen 300 is 15 cm and the vertical width is 7 cm, the horizontal width of the first and second reproduced images 311 and 312 in FIG. 14 is about 6 cm, and the vertical width is about 6 cm. The width can be about 4 cm.

After the screen is enlarged, in the case of FIG. 15 , the size of the first reproduced image 311 may be maintained, and the width of the second reproduced image 312 may be about 6.89 cm and the vertical width may be about 5 cm.

Of course, this can also be set as a ratio. For example, if it is assumed that the first reproduced image 311 is 100%, the second reproduced image 312 may be 110%.

Also, the first reproduced image 311 may be reduced by 90% and the second reproduced image 312 may be enlarged by 140%. That is, when it is difficult to distinguish because there are a large number of people displayed in the second reproduced image 312 , it is enlarged beyond the boundary line 303 . Of course, in this case, the first reproduced image 311 is reduced.

In FIG. 14 , it is assumed that the camera 121 is installed in the communication terminal 20 in the longitudinal direction, but when two cameras ( 121 in FIG. 2 ) are installed at regular intervals on the left and right sides of the communication terminal 20 , top and bottom It is possible to move the communication terminal 20 in an upward direction under or under the direction. In this case, the first reproduced image 311 and the second reproduced image 312 may be simultaneously displayed and reproduced.

In addition, the steps of the method or algorithm described in relation to the embodiments disclosed herein are implemented in the form of program instructions that can be executed through various computer means such as a microprocessor, a processor, a CPU (Central Processing Unit), etc. It can be recorded on any available medium. The computer-readable medium may include program (instructions) codes, data files, data structures, etc. alone or in combination.

The program (instructions) code recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs, DVDs, Blu-rays, and the like, and ROM, RAM ( A semiconductor memory device specially configured to store and execute program (instruction) code such as RAM), flash memory, and the like may be included.

Here, examples of the program (instruction) code include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes such as those generated by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

1: Video playback system
11,12: first and second photo cards
11-1,12-1: first and second photo images
20: communication terminal
30: communication network
40: management server
41: database
110: wireless communication unit 120: A/V (Audio/Video) input unit
130: user input unit 140: sensing unit
160: memory 170: interface unit
180: control unit 190: power supply unit
300: display screen
301: background
311: first content information (played video)
312: second content information (played video)

Claims (5)

delete delete delete (a) As the communication terminal 20 moves on the surface of the two photo cards 11 and 12 placed side by side at regular intervals, two photo images disposed on the two photo cards 11 and 12, respectively ( 11-1, 12-1) and generating two pieces of photographed image information corresponding to each;
(b) the management server 40 searching for two pieces of content information 311 and 312 respectively matching the two pieces of feature point information using two pieces of feature point information extracted from each of the two pieces of captured image information;
(c) transmitting, by the management server 40, the two pieces of content information 311 and 312 found to the communication terminal 20; and
(d) the first face size of the first specific person 311-1 and the second face of the second specific person 312-1 that the communication terminal 20 displays in the two pieces of the content information 311 and 312, respectively comparing the sizes with each other, enlarging any one or both of the two pieces of content information 311 and 312 according to the comparison result and outputting them on the display screen 300;
Step (d) is,
(d-1) displaying, by the communication terminal 20, the two pieces of the content information 311 and 312 on the display screen 300 in a split screen;
(d-2) In the communication terminal 20, the first specific person 311-1 and the second specific person 312-1 have a predetermined width on the split screen left and right from the center line of the split screen, respectively. determining whether it is within a specific location with
(d-3) According to the determination result, the communication terminal 20 sets the first focusing area 311-2 and the second specific object 312-1 for the first specific person 311-1. generating the first face size and the second face size through a second focusing area (312-2) for ;
(d-4) comparing the first face size with the second face size by the communication terminal 20; and
(d-5) outputting, by the communication terminal 20, on the display screen 300 by magnifying any one of the two pieces of content information 311 and 312 according to the comparison result How to play video using a photo card. delete

Images