KR102105969B1 - User device for receiving information regarding image and method thereof - Google Patents

Abstract

The user device receiving the information related to the image includes an image capturing unit that captures all or part of the image using an image sensor, an area determining unit that determines which of the plurality of areas included in the image is captured, and the captured area. It may include an information requesting unit for requesting the corresponding information from the service providing device, and an information receiving unit receiving pre-mapped information corresponding to the captured area from the service providing device.

Description

USER DEVICE FOR RECEIVING INFORMATION REGARDING IMAGE AND METHOD THEREOF}

The present invention relates to a user device and method for receiving information related to an image captured through a user device.

With the development of information and communication technology, mobile phones and so-called smart phones that integrate mobile phone functions and data communication functions have emerged. Unlike traditional mobile phones, which were released as finished products and used only given functions, smartphones can install and use hundreds or thousands of various applications (applications) as desired by the user. Users have come to use smartphones.

As an example of various applications, a user of a smartphone may recognize a printed image (for example, an image included in a magazine or product packaging) using an image recognition application and receive image-related information from a service provider.

It is intended to provide a user device and method for receiving information related to an image using a camera (image sensor) provided or connected to a user device. However, the technical problems to be achieved by the present embodiment are not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, the user device according to the first aspect of the present invention, an image capture unit for capturing all or part of the image using an image sensor, a plurality of areas included in the image An area determining unit that determines which of the areas is captured, an information requesting unit requesting information corresponding to the captured area from a service providing device, and receiving information pre-mapped corresponding to the captured area from the service providing device It characterized in that it comprises an information receiving unit.

Meanwhile, a method of receiving information related to an image according to a second aspect of the present invention includes capturing all or part of the image using an image sensor, and which region of a plurality of regions included in the image is captured And determining, requesting information corresponding to the captured area from a service providing device, and receiving pre-mapped information corresponding to the captured area from the service providing device.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, there may be additional embodiments described in the drawings and detailed description of the invention.

According to any one of the above-described problem solving means of the present invention, the user device may capture an image using an image sensor, receive information related to the captured image from a service providing apparatus, and provide it to a user.

1 is a block diagram of a system for providing content information according to an embodiment of the present invention.
2 is a configuration diagram of the user device shown in FIG. 1 according to an embodiment of the present invention.
3 is an example of content shown in FIG. 1 according to an embodiment of the present invention.
4 is an example of a user interface displayed on the user device shown in FIG. 1 according to an embodiment of the present invention.
5 is an operation flowchart illustrating a method of providing image-related information according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts irrelevant to the description are omitted, and like reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element in between. . Also, when a part is said to “include” a certain component, it means that the component may further include other components, not to exclude other components, unless otherwise stated. However, it should be understood that the existence or addition possibilities of numbers, steps, actions, components, parts or combinations thereof are not excluded in advance.

In the present specification, the term “unit” includes a unit realized by hardware, a unit realized by software, and a unit realized by using both. Further, one unit may be realized by using two or more hardware, and two or more units may be realized by one hardware.

Some of the operations or functions described in this specification as being performed by a terminal or device may be performed instead on a server connected to the corresponding terminal or device. Similarly, some of the operations or functions described as being performed by the server may be performed in a terminal or device connected to the corresponding server.

Hereinafter, specific contents for carrying out the present invention will be described with reference to the accompanying drawings or processing flow charts.

1 is a block diagram of a system for providing content information according to an embodiment of the present invention. Referring to FIG. 1, the content information providing system may include a user device 100, content 200, a captured content image 201, a service providing device 300, and a network 400. However, since the content information providing system of FIG. 1 is only an embodiment of the present invention, the present invention is not limitedly interpreted through FIG. 1.

The user device 100 may capture all or part of an image included in the content 200 by using an image sensor, and determine which region among a plurality of regions included in the image is captured. Here, the image sensor may be an imaging circuit element included in the user device 100. For example, the image sensor may be a CMOS image sensor, a CCD image sensor, or the like.

The user device 100 requests information corresponding to the captured content image 201 area from the service providing apparatus 300, and pre-mapped information corresponding to the captured content image 201 area as image related information, It can be received from the service providing device 300. Here, the captured content image 201 may be a plurality of objects or text included in the content 200.

The method of obtaining the image-related information may be performed through an application program installed and executed in the user device 100 to provide a corresponding service.

The user device 100 may include a mobile terminal capable of wireless communication, and according to various embodiments of the present invention, the user device 100 may have various forms. For example, the user device 100 may be a portable terminal capable of accessing a remote server through a network. Here, as an example of a portable terminal, as a wireless communication device in which portability and mobility are guaranteed, Personal Communication System (PCS), Global System for Mobile communications (GSM), Personal Digital Cellular (PDC), Personal Handyphone System (PHS), PDA (Personal Digital Assistant), International Mobile Telecommunication (IMT) -2000, Code Division Multiple Access (CDMA) -2000, W-Code Division Multiple Access (W-CDMA), Wireless Broadband Internet (Wibro) terminal, smartphone (smartphone) , All kinds of handheld based wireless communication devices such as tablet PCs may be included.

The content 200 may exist in various forms. For example, the content 200 may be in the form of a credit card-sized plastic card, or may be in the form of a sticker attached to an object or the like printed on an object. In FIG. 1, the content 200 is illustrated in the form of a card, but the form of the content 200 is not limited thereto.

The content 200 may include an image inserted or printed. Here, the image may be marked with watermarking or fingerprinting. Also, the image may include a plurality of objects and text.

According to an embodiment of the present invention, the content 200 may include a QR code or NFC tag containing image-related information. In this case, the user of the user device 100 can obtain image-related information from the QR code or NFC tag using the user device 100 having a QR code or NFC tag reader.

The captured content image 201 is an image of the content 200 captured through the image sensor provided in the user device 100, and may be displayed on the display device of the user device 100. Also, the captured content image 201 may be an entire image included in the content 200 or a partial image.

The service providing apparatus 300 may receive information corresponding to the captured content image 201 from the user device 100 and transmit information mapped to the captured content image 201 to the user device 100. have. In this case, information corresponding to the captured image area may be searched from the DB of the service providing device 300, but is not limited thereto.

Each of the components of FIG. 1 constituting the content information providing system may be connected through the network 400. For example, as illustrated in FIG. 1, each of the user device 100, the card-type content 200, and the service providing device 300 may be connected through the network 400. The network 400 refers to a connection structure capable of exchanging information between nodes such as terminals and servers, and examples of the network 400 include the Internet, a local area network (LAN), and wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), and the like, but are not limited thereto.

Hereinafter, the operation of the user device 100 of FIG. 1 will be described in more detail.

2 is a configuration diagram of a user device 100 shown in FIG. 1 according to an embodiment of the present invention. Referring to FIG. 2, the user device 100 may include an image capture unit 110, an area determining unit 120, a user interface 130, an information requesting unit 140, and an information receiving unit 150. However, the user device 100 illustrated in FIG. 2 is only one implementation example of the present invention, and various modifications are possible based on the components illustrated in FIG. 2. For example, the user device 100 may further include a user interface for receiving certain commands or information from the user. In this case, the user interface may generally be an input device such as a keyboard or a mouse, but may also be a graphical user interface (GUI) expressed on a video display device.

The image capture unit 110 may capture all or part of an image included in the content 200 using an image sensor. At this time, the image includes an object or text, and a part of the captured image may correspond to any one of the object or text. Here, the object may be a person or a product. In addition, the image at this time may be that watermarking or fingerprinting is displayed to prevent forgery. Watermarking or fingerprinting may protect image copyright by inserting image-related information into an image included in the content 200. Here, watermarking or fingerprinting is a technique of recording copyright information in a form unknown to a user in content such as an image or music.

The region determination unit 120 may determine which region of a plurality of regions included in the image is captured. Here, each of the plurality of regions included in the image may be mapped with different information. For example, profile information corresponding to the person, sound source associated with the person, and fandom product information may be mapped to each of the plurality of persons included in the image.

The user interface 130 may include a preview area displaying an external scene reflected on the image sensor. Furthermore, the user interface 130 may include a category UI (User Interface) for selecting a category corresponding to each of a plurality of areas included in the image to be captured in advance. In addition, the user interface 130 may include a category UI (User Interface) that pre-selects a category corresponding to each of a plurality of information mapped to an image to be captured. Here, the category may include a text category and an image category. Also, the category may include an entire image category and some image categories.

The information requesting unit 140 may request information corresponding to the captured area from the service providing device 300. For example, when the user device 100 captures all the images, the information requesting unit 140 requests the service providing apparatus 300 for information such as video content, image content, or music content corresponding to the captured image. You can.

As another example, when the user device 100 captures a product included in an image, the information requesting unit 140 may request the service providing apparatus 300 for information about the captured product.

The information receiving unit 150 may receive pre-mapped information corresponding to the captured region from the service providing apparatus 300. For example, when all of the images are captured, the information receiving unit 150 may receive video content, image content, or music content corresponding to the entire captured image from the service providing device 300. Furthermore, when the pre-mapped information is music content, the information receiving unit 150 may receive a message from the service providing device 300 that activates a music playback program that plays music content together with the music content.

As another example, when a person included in the image is captured, the information receiving unit 150 may receive profile information on the captured person from the service providing device 300.

In addition, when a product included in the image is captured, the information receiving unit 150 may receive information on the captured product from the service providing device 300. In addition, when the text included in the image is captured, the information receiving unit 150 may receive information about a fandom product corresponding to the captured text from the service providing device 300.

3 is an example of the content 200 shown in FIG. 1 according to an embodiment of the present invention. Referring to FIG. 3, the content 200 may be card-shaped content including images and text. At this time, as described above, the images and texts included in the content 200 may be stored in the service providing apparatus 300 in advance, such as images, music, and products corresponding to the images and texts.

If the person included in the image of the content is 'Big Bang' members, the user uses the user device 100 to capture one or more of the entire image 32, a specific person 33 of the 'Big Bang' members, and text 34 can do. However, the capture target is not limited to the entire image 32, the specific person 33, and the text 34, and may further include clothes, shoes, accessories, and the like worn by each person.

When the user of the user device 100 captures the entire image 32 of the 'Big Bang' member in the content 200 using the user device 100, the user device 100 is configured for the captured image or the captured image. Information may be transmitted to the service providing device 300, and in response, the service providing device 300 may capture the video content, image content, and music content related to 'Big Bang' as the captured image to the user device 100 Can provide.

When a user of the user device 100 captures a specific person 33 among 'big bang' members in the content 200 using the user device 100, the user device 100 uses the captured image or the captured image. Information on the service can be transmitted to the service providing device 300, and in response, the service providing device 300 can provide the user device 100 with profile information on the captured person. At this time, the profile information may include information such as the real name, age, date of birth, affiliated company, and awards.

When the user of the user device 100 captures the text 34 marked as 'BIGBANG IS BACK' in the content 200 using the user device 100, the user device 100 captures the captured image or the captured image Information on the service providing device 300 may be transmitted, and in response, the service providing device 300 may provide the user device 100 with information about a fandom product corresponding to the captured text. have. Fandom products can be pecure, clothing, cultural content products, etc. that correspond to each 'Big Bang' member.

Examples of receiving information related to the captured image mentioned above, when the entire image is captured, video content, image content, music content, etc. related to the object of the content; When some person images are captured, profile information including information such as the real name, age, date of birth, agency, and award for the captured person; When the text is captured, it has been described a case where information on a fandom product corresponding to the captured text is provided, but this is only various embodiments of the present invention, and information provided in correspondence with the captured image is described. It is not limited. Specifically, the information provided corresponding to the captured image may be information that is previously stored by matching the image captured by the service providing apparatus 100.

Furthermore, when the user of the user device 100 captures clothes, shoes, accessories, etc. worn by each person in the content 200 using the user device 100, the user device 100 captures the captured image or Information about the image may be transmitted to the service providing device 300, and in response, the service providing device 300 may provide object information (for example, the name of the product, corresponding to the captured clothing, shoes, accessories, etc.). Brand, price, preferred age, product image, etc.) to the user device 100.

4 is an example of a user interface displayed on the user device 100 shown in FIG. 1 according to an embodiment of the present invention. Referring to FIG. 4, the user interface includes a preview area 41, a description area for a selected category button (hereinafter referred to as a 'button description area') 42, and category buttons A to D corresponding to the image area to be captured. (43 to 46).

When the user of the user device 100 selects the category button A 43, a description corresponding to the category button A 43 may be displayed in the button description area 42. For example, when the user touches the category button A 43, a description of “Please recognize the entire image” may be displayed in the button description area 42. In this case, the user adjusts the position of the user device 100 to display the entire image included in the content 200 in the preview area 41 according to the output description, and the corresponding entire image displayed in the preview area 41 Can be captured. At this time, in order to capture the image, the user may touch a specific physical button for capturing the image or a UI button displayed on the display. On the other hand, when the entire image is displayed in the preview area 41, the user device 100 displays , By recognizing this, and controlling the image sensor and corresponding hardware / software, it is possible to capture an image without user input. This may be the case when not only the entire image but also a partial image corresponding to each category button is displayed.

After capturing the entire captured image, the user device 100 may transmit the captured image or information about the captured image to the service providing apparatus 300, and in response, the user device 100 may capture the captured image The movie or album contents corresponding to the content may be provided from the service providing device 300 and output to the preview area 41.

When the user of the user device 100 selects the category button B 44, a description corresponding to the category button B 44 may be displayed in the button description area 42. When the description "Please recognize the person" is output in the button description area 42, in response to this, the user may display a specific person image included in the content 200 in the preview area 41 according to the output description. The position of the user device 100 can be adjusted to be displayed, and a specific person image displayed in the preview area 41 can be captured. After capturing the captured specific person, the user device 100 may transmit information about the captured specific person image or the captured specific person image to the service providing apparatus 300, and in response thereto, the user device 100 May receive a profile corresponding to a captured specific person image, a movie in which a specific person appeared, and existing album information from the service providing device 300 and output the information to the preview area 41.

When the user of the user device 100 selects the category button C 45, a description corresponding to the category button C 45 may be output in the button description area 42. When the description of "Please recognize the entire image" is output in the button description area 42, in response to this, the user may display the entire image included in the content 200 in the preview area 41 according to the output description. The position of the user device 100 can be adjusted to be displayed, and the entire image displayed in the preview area 41 can be captured. After capturing the entire captured image, the user device 100 may transmit the captured image or information about the captured image to the service providing apparatus 300, and in response, the user device 100 may capture the captured image Movie and album production notes, interviews, and unpublished contents corresponding to (eg, a director's edition, making film, etc.) may be provided from the service providing device 300 and output to the prev area 41.

When the user of the user device 100 selects the category button D 46, a description corresponding to the category button D 46 may be output in the button description area 42. When the description "Please recognize the character" is output in the button description area 42, in response to this, the user displays the text image included in the content 200 in the preview area 41 according to the output description. It is possible to adjust the position of the user device 100 and capture the corresponding text image displayed in the preview area 41. After capturing the captured text, the user device 100 may transmit information about the captured text image or the captured text image to the service providing apparatus 300, and in response, the user device 100 captures the captured text image Information about a fandom product corresponding to a text image may be provided from the service providing device 300 and output to the preview area 41. As examples of determining the target of the image to be captured corresponding to the category buttons A to D (43 to 46) mentioned above, the entire image of the content 200 is captured in response to the category button A (43); Capture a specific person image of the content 200 corresponding to the category button B 44; Capture the entire image of the content 200 corresponding to the category button C 45; Although the case of capturing a text image of the content 200 in response to the category button D 46 has been described, this is only various embodiments of the present invention, and the target of the image to be captured corresponding to each category button is limited thereto. It does not become. Specifically, a target of an image to be captured corresponding to each category button may be set by a service provider or service providing device 300 that distributes an application program that provides a corresponding service.

5 is an operation flowchart illustrating a method of providing image-related information according to an embodiment of the present invention. The method for providing image-related information according to the embodiment shown in FIG. 5 includes steps performed in time series in the user device 100 according to the embodiments shown in FIGS. 1 to 4. Therefore, even if it is omitted below, the contents described with respect to the user device 100 of FIGS. 1 to 4 also apply to a method for providing image-related information according to the embodiment illustrated in FIG. 5.

Referring to FIG. 5, in step S501, the user device 100 selects a user input to select a category through a category UI (User Interface) button corresponding to each of a plurality of areas included in an image to be captured. ). Furthermore, the user device 100 may capture all or part of the image using an image sensor based on the received user input.

In step S502, the user device 100 may determine which area of the plurality of areas included in the image is captured.

In step S503, the user device 100 may request information corresponding to the captured area from the service providing apparatus 300.

In operation S504, the user device 100 may receive pre-mapped information corresponding to the captured region from the service providing apparatus 300 as image-related information. At this time, information received from the service providing apparatus 300 may be displayed on the display apparatus of the user device 100. When the information received from the service providing device 300 includes music content, the received information may include a message for activating a music playback program that plays music content. The music playback program may be activated based on the received message. Music content can be played through a playback program.

In the above description, steps S501 to S504 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted if necessary, and the order between the steps may be changed.

One embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and includes any information delivery media.

The above description of the present invention is for illustration only, and a person having ordinary knowledge in the technical field to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be interpreted to be included in the scope of the present invention. .

100: user device
200: card type content
300: service providing device

Claims (19)

In the user device receiving information related to the image included in the content,
An image capture unit capturing all or part of the image using an image sensor;
An area determining unit determining which area of the plurality of areas included in the image is captured;
An information requesting unit requesting information corresponding to the captured area from a service providing device; And
An information receiving unit receives pre-mapped information corresponding to the captured area from the service providing device,
The area determining unit determines whether an area captured from the image is an entire area including all of the plurality of areas or a partial area of the plurality of areas,
The information received from the service providing apparatus is determined based on whether an area captured from the image is an entire area including all of the plurality of areas or a partial area of the plurality of areas.
According to claim 1,
Each of the plurality of regions included in the image is mapped with different information, a user device.
According to claim 1,
A user device further comprising a user interface including a category UI (User Interface) for preselecting a category corresponding to each of a plurality of areas included in the image to be captured.
The method of claim 3,
The user interface includes a preview area displaying an external scene reflected on the image sensor.
The method of claim 3,
The category includes a text category and an image category, a user device.
The method of claim 3,
The category includes all image categories and some image categories.
According to claim 1,
A user device further comprising a user interface including a category UI (User Interface) for preselecting a category corresponding to each of a plurality of information mapped to the image to be captured.
According to claim 1,
The image includes a plurality of people, and a part of the captured image corresponds to any one of the plurality of people.
According to claim 1,
When all of the images are captured, the information received from the service providing device includes video content, image content or music content corresponding to the image.
The method of claim 9,
The information received from the service providing device includes a message for activating a music playing program that plays the music content, a user device.
According to claim 1,
When a person included in the image is captured, the information received from the service providing device includes profile information for the captured person.
According to claim 1,
When a product included in the image is captured, the information received from the service providing device includes information on the captured product.
According to claim 1,
When the text included in the image is captured, the information received from the service providing device includes information on a fandom product corresponding to the captured text.
According to claim 1,
The image is watermarking or fingerprinting is displayed, the user device.
In the method of receiving information related to the image included in the content,
Capturing all or part of the image using an image sensor;
Determining which region of the plurality of regions included in the image is captured;
Requesting information corresponding to the captured area from a service providing device; And
And receiving pre-mapped information corresponding to the captured area from the service providing apparatus,
The determining includes determining whether an area captured from the image is an entire area including all of the plurality of areas or a partial area of the plurality of areas,
The information received from the service providing device is determined based on whether an area captured from the image is an entire area including all of the plurality of areas or a partial area of the plurality of areas. .
The method of claim 15,
And receiving a user input for selecting any one category through a category UI (User Interface) that displays a category corresponding to each of a plurality of areas included in the image to be captured.
The method of claim 15,
And displaying information received from the service providing device on a display device of a user device.
The method of claim 15,
When the information received from the service providing device includes music content, the received information further includes a message for activating a music playback program for playing the music content.
The method of claim 18,
Activating the music playback program based on the received message; And
And reproducing the music content through the music reproduction program.

Images