KR20170078109A - Method and Apparatus for Providing Recommended Contents - Google Patents

Abstract

The present invention relates to a method and apparatus for providing recommended contents, which extracts fact information using sensor information and data analysis information of a photograph taken or captured by a user, and extracts estimated information using context information obtained by analyzing photographs Extracts recommendation information necessary for the user based on the fact information and the presumed information, and provides the user with the customized content according to the change of the behavior of the user and the surrounding situation while managing the pictures more variously and efficiently can do.
In particular, one or more predetermined criteria may be set based on the extracted fact information and the estimated information, and one or more pieces of information including recommendation information for classifying the pictures according to the set criteria and extracting them from the classified or classified pictures Providing the user with customized content tailored to the user's situation and providing the third service provider with one or more pieces of information that can be extracted from the classified criteria or the classified photos to provide an effective marketing strategy based on age, You can help them perform.

Description

TECHNICAL FIELD The present invention relates to a method and apparatus for providing recommended contents,

The present invention relates to a method and apparatus for providing recommended contents. More particularly, the present invention relates to a method and apparatus for providing recommended contents by extracting fact information through sensor information and image analysis information of a photograph taken or captured by a user, extracting estimated information through context analysis information, And a method and an apparatus for classifying a plurality of image data according to a predetermined criterion on the basis of the fact information and the estimation information to provide customized recommendation contents specialized for the user.

The contents described in this section merely provide background information on the present embodiment and do not constitute the prior art.

Thanks to the development of Information Communication Technology (ICT), a variety of new industries have emerged based on these ICTs, and they have further intelligentized / networked existing products and services.

Particularly, in the ICT field in recent years, big data, mobile, wearable technologies have become a hot topic, and they are able to replace all services with data and provide them as customized and personalized services, It becomes the technological foundation to enter the era of personal from the age of mass.

Mobile technology is the basis for connecting and communicating anytime and anywhere beyond the limits of time and space. Wearable technology can be applied to various types of devices such as wearing, bending, and spreading beyond the limits of conventional devices .

Big Data is a new service that serves as a catalyst for creating a new paradigm in ICT. Big Data is a vast amount of data that could not be handled by existing management and analysis systems. As the technology emerged, big data began to attract attention in the social and economic fields, and studies related to large-scale data related technologies and tools (collection, storage, search, sharing, analysis, visualization) With the rapid development of ICT, the Big Data became more influential in combination with social, objects, and data as the smart age became commonplace.

In addition, the emergence of mobile, cloud, and SNS has changed the underlying infrastructure and data, and the view of data has changed. Particularly, responding to data explosion and data analysis have become important issues of ICT, Data is recognized as a key issue in the ICT market and technology development.

Particularly, since 2010, the proliferation of smart and mobile, the spread of SNS usage such as Facebook and Twitter, and the change of communication method have resulted in the increase of data generated by users voluntarily participating.

As a concrete example, the combination of the camera function of the smartphone and the SNS service allows the users to take various pictures (surrounding scenes, restaurants, signboards, schedules, products) To share and keep it with others.

Therefore, the number of photographed data to be photographed and shared or stored by the users is rapidly increasing, and there is a need for an efficient management and storage system for them.

In this regard, there is a service for managing the photographs based on the position by recording the position information at the time of photographing at the time of photographing in the metadata of the photograph, but this is a method for sorting the photographs based on the position only, There is a problem in that it is not appropriate to manage a photograph used as a photograph.

 Accordingly, there has been a demand for a storage platform and service of a new point of view that provides a service suited to the change of the behavior of the user and the surrounding situation in order to meet the user's needs in accordance with the change of the behavior of the user.

Korean Patent Laid-Open No. 10-2007-0060617 (Title: Method and apparatus for performing functions in a portable terminal equipped with a camera, 2007.06.13)

In order to meet the above-mentioned demand, the present invention is a method for receiving a photograph taken or captured by a user, the content providing server receiving and storing the photograph, extracting fact information through sensor information and image analysis information of the received photograph, Extracts the estimation information based on the analyzed information, extracts the recommendation information suitable for the user's situation based on the extracted fact information and the estimated information, and provides the extracted information to the user, And to provide a method and apparatus for providing recommended contents that can provide customized contents to a user.

In particular, one or more predetermined criteria may be set based on the extracted fact information and the estimated information, and one or more pieces of information including recommendation information for classifying the pictures according to the set criteria and extracting them from the classified or classified pictures Providing the user with recommended content suitable for the user's situation and providing the third service provider with one or more pieces of information that can be extracted from the classified criteria or the classified pictures to provide an effective marketing strategy based on age, And to provide a content providing method and apparatus that allow a user to carry out the present invention.

However, the object of the present invention is not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood from the following description.

According to another aspect of the present invention, there is provided a method of providing content, the method comprising: receiving and storing at least one image data and sensor information corresponding to each image data from at least one terminal device, Extracting image analysis information by analyzing a character or graphic included in each of the image data based on the image data, analyzing context information indicating a situation included in each of the image data based on the image data, Extracting fact information on each of the image data based on the image analysis information and the sensor information, extracting estimation information on each of the image data based on the analyzed context information, Of fact information and inferred information Classifying the at least one image data according to at least one criterion based on one or more criteria and transmitting at least one information relating to the at least one classified image data to the at least one terminal device or the third service server .

The method may further include extracting one or more recommendation information based on the fact information and the estimation information of the classified image data, and the extracting of the recommendation information may include extracting the extracted fact information and the estimated information, Generating a recommendation list including at least one recommendation information, and extracting recommendation information other than recommendation information corresponding to the image analysis information from at least one recommendation information included in the recommendation list.

The method may further include extracting characteristic information for each user from the classified image data based on at least one of the fact information and the estimation information prior to the transmitting step, And providing at least one image data to the terminal apparatus.

The classifying step includes a step of classifying the image data associated with each other according to a certain criterion on the basis of the fact information, a step of classifying the image data classified as the primary image based on the estimated information, And sorting.

Further, the method may further include mapping and storing at least one of the fact information and the estimation information for each of the terminal devices with the image data, and the sensor information may include at least one of At least one of the location information, the date information, and the weather information of the terminal device.

Meanwhile, the content providing method may be provided as a computer-readable recording medium on which a program for executing the program is recorded, and may be provided as a computer program stored in a computer-readable recording medium which is implemented to execute the program.

According to another aspect of the present invention, there is provided a content providing server including a communication module for transmitting and receiving data to and from at least one terminal device and a third service server in cooperation with a communication network, Extracting image analysis information by analyzing characters or graphics included in each of the image data based on the image data stored in the storage module, Extracting fact information based on the analyzed image analysis information and the sensor information, extracting estimation information based on the context information, analyzing context information indicating a situation included in each of the image data based on the image data, , The extracted fact information and the recommendation information A control unit configured to classify the at least one image data based on the at least one image data and to transmit at least one information associated with the classified at least one image data to the at least one terminal device or the third service server via the communication module Modules.

At this time, the control module can extract one or more recommendation information based on the fact information and the estimation information of the classified image data, and the recommendation information including one or more recommendation information based on the extracted fact information and the estimated information And extracts the recommendation information other than the recommendation information corresponding to the image analysis information from the one or more recommendation information included in the recommendation list.

Also, the control module may extract characteristic information for each user from the classified image data based on at least one of the fact information and the estimation information, and the control module may include at least one of the at least one And provide the image data to the terminal device.

In addition, the control module firstly classifies related image data according to a certain criterion on the basis of the fact information, and secondarily classifies the first classified image data into image data having a predetermined characteristic based on the estimated information And may control at least one of the fact information and the estimation information for each of the terminal devices to be mapped to the image data and stored in the storage module.

The sensor information may include at least one of position information, date information, and weather information of the terminal at the time when the terminal device generates image data.

According to the present invention, fact information is extracted using sensor information and data analysis information of a photograph taken or captured by a user, extracting estimation information using context information obtained by analyzing photographs, By extracting the recommendation information necessary for the user based on the information and providing the same to the user, it is possible to provide the customized content according to the change of the behavior of the user and the surrounding situation while managing the pictures more diversely and efficiently.

In particular, one or more predetermined criteria may be set based on the extracted fact information and the estimated information, and one or more pieces of information including recommendation information for classifying the pictures according to the set criteria and extracting them from the classified or classified pictures Providing the user with customized content tailored to the user's situation and providing the third service provider with one or more pieces of information that can be extracted from the classified criteria or the classified photos to provide an effective marketing strategy based on age, You can help them perform.

In addition, various effects other than the above-described effects can be directly or implicitly disclosed in the detailed description according to the embodiment of the present invention to be described later.

1 is a diagram illustrating a configuration of a content providing system according to an embodiment of the present invention.
2 is a block diagram showing a configuration of a terminal device according to the present invention.
3 is a flowchart illustrating an operation procedure of a terminal according to an embodiment of the present invention.
4 is a block diagram showing a configuration of a content providing server according to the present invention.
5 to 7 are flowcharts illustrating an operation procedure of a content providing server according to an embodiment of the present invention.
8 to 11 are illustrations for explaining various embodiments of the present invention.
12 is a diagram illustrating an operating environment in a service system for providing contents according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS For a more complete understanding of the nature and advantages of the present invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings, in which:

In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted. It should be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings.

The terms and words used in the following description and drawings are not to be construed in an ordinary sense or a dictionary, and the inventor can properly define his or her invention as a concept of a term to be described in the best way It should be construed as meaning and concept consistent with the technical idea of the present invention.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It is to be understood that equivalents and modifications are possible.

Also, terms including ordinal numbers such as first, second, etc. are used to describe various elements, and are used only for the purpose of distinguishing one element from another, Not used.

For example, without departing from the scope of the present invention, the second component may be referred to as a first component, and similarly, the first component may also be referred to as a second component.

Also, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

The singular expressions include plural expressions unless the context clearly dictates otherwise. It is also to be understood that the terms such as " comprising "or" having ", as used herein, are intended to specify the presence of stated features, integers, It should be understood that the foregoing does not preclude the presence or addition of other features, numbers, steps, operations, elements, parts, or combinations thereof.

Also, the terms "part," "module," and the like, which are described in the specification, refer to a unit for processing at least one function or operation, and may be implemented by hardware or software or a combination of hardware and software.

Also, the terms " a or ", "one "," the ", and the like are synonyms in the context of describing the invention (particularly in the context of the following claims) May be used in a sense including both singular and plural, unless the context clearly dictates otherwise.

In addition to the above-described terms, certain terms used in the following description are provided to facilitate understanding of the present invention, and the use of such specific terms may be changed to other forms without departing from the technical idea of the present invention.

In addition, embodiments within the scope of the present invention include computer-readable media having computer-executable instructions or data structures stored on computer-readable media.

Such computer-readable media can be any available media that is accessible by a general purpose or special purpose computer system. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or in the form of computer- But is not limited to, a physical storage medium such as any other medium that can be used to store or communicate certain program code means of the general purpose or special purpose computer system, .

In addition, the invention may be practiced with other computer systems, including personal computers, laptop computers, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, a pager, and the like. < RTI ID = 0.0 > [0040] < / RTI >

The invention may also be practiced in distributed systems environments where both local and remote computer systems linked by a combination of wired data links, wireless data links, or wired and wireless data links over a network perform tasks. In a distributed system environment, program modules may be located in local and remote memory storage devices.

On the other hand, in the description of the embodiment according to the present invention, it is assumed that the image or image data means photograph or photograph data.

However, the image or image data according to the exemplary embodiment of the present invention does not necessarily mean only photograph or photograph data, and any image such as a picture or a shape that can be recognized as an image other than a photograph may be any image according to an embodiment of the present invention Image or image data.

Now, an apparatus for providing content in a content providing system according to an embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram for schematically explaining a structure of a content providing system according to an embodiment of the present invention.

Referring to FIG. 1, a content providing system according to an embodiment of the present invention may include a terminal device 200, a content providing server 300, and a third service server 400.

Here, the content providing server 300 is interworked with the terminal device 200 and the third service server 400 through the communication network 100.

Each component will now be schematically described with reference to FIG.

The communication network 100 is responsible for transferring data between the terminal device 200 and the content providing server 300 or between the third service server 400 and the content providing server 300, Wired communication methods such as Ethernet, xDSL (ADSL, VDSL), HFC (Hybrid Fiber Coaxial Cable), FTTC (Fiber to the Curb) and FTTH (Fiber To The Home) Wireless LAN (WLAN), Wi-Fi, WiBro, WiMAX, HSDPA, LTE and LTE-Long Term Evolution Advanced Method.

In addition, the communication network 100 may include a plurality of access networks (not shown) and a core network (not shown), and may include an external network such as an Internet network (not shown).

Here, the access network (not shown) is an access network that performs wired / wireless communication with the terminal device 200 and includes a plurality of base stations such as a base station (BS), a base transceiver station (BTS), a NodeB, an eNodeB, (Base Station Controller), and an RNC (Radio Network Controller).

Also, as described above, the digital signal processing unit and the radio signal processing unit integrally implemented in the base station are divided into digital units (hereinafter, referred to as DUs and radio units (RUs) A plurality of RUs (not shown) may be provided in each of a plurality of areas, and a plurality of RUs (not shown) may be connected to a centralized DU (not shown).

A core network (not shown) constituting a mobile network together with an access network (not shown) performs a role of connecting an access network (not shown) and an external network (not shown), for example, an Internet network (not shown).

As described above, the core network (not shown) is a network system that performs main functions for mobile communication services such as mobility control and switching between access networks (not shown), and includes a circuit switching or a packet exchange and manages and controls the packet flow in the mobile network. The core network (not shown) manages inter-frequency mobility and plays a role for interworking with traffic in an access network (not shown) and a core network (not shown) and other networks such as the Internet It is possible.

Such a core network (not shown) may further include an SGW (Serving Gate Way), a PGW (PDN GateWay), an MSC (Mobile Switching Center), a HLR (Home Location Register), a MME (Mobile Mobility Entity) .

The Internet network (not shown) refers to a public network, that is, a public network, in which information is exchanged according to the TCP / IP protocol. The Internet network is connected to the terminal apparatus 200, (Not shown) and an access network (not shown) to the content providing server 300, and conversely, the information provided to the content providing server 300 can be provided to the core network (not shown) and the access network To the terminal device 200 via the Internet. However, the present invention is not limited thereto, and the content providing server 300 may be implemented integrally with a core network (not shown).

In addition to the above-described communication methods, other widely known or later-developed communication methods may be included.

The terminal device 200 of the present invention means a user device capable of transmitting and receiving various data via the communication network 100 according to a user's operation. The terminal device 200 can perform voice or data communication through the communication network 100 and can exchange information with the content providing server 300 through the communication network 100.

The terminal device 200 of the present invention may include a memory for storing programs and protocols for transmitting and receiving image data, a microprocessor for executing and controlling various programs, and the like.

Such a terminal device 200 is a service application for accessing an application providing device (not shown) connected to the communication network 100, for example, an application store, analyzing image data from the application store, and providing recommended contents to the user Can be received and installed. The user terminal device 200 executes the service application and transmits the image data to the content providing server 300 so that the content provided by the service application 300 from the content providing server 300 (for example, Provision of recommended contents suitable for the user) and provide the user with the contents.

According to the embodiment of the present invention, the terminal device 200 captures an image of a photograph or the like by a user's request, or captures a photograph that is photographed or generated by another user or the user and uploaded to an SNS, an instant message, an e-mail or the like.

At this time, when the image is photographed or generated, sensor information such as position information, date information, weather information, and the like, on which the image is photographed or generated, may be included in the image data, and may be included in the metadata of the image data .

However, the sensor information including the position information, the date information, and the weather information may not be included in the image data but may be separately generated and managed.

The location information may be acquired through a GPS (Global Positioning System) of the terminal device 200, and the time information may include time information received from the terminal device 200 by the terminal device 200, The weather information can be generated based on the sensing value measured by the ambient light sensor or the temperature sensor mounted on the terminal device 200 and can be interlocked with the weather application installed in the terminal device 200 And may be obtained from the weather application.

The terminal device 200 captures or captures image data including location information, etc., and transmits the image data together with user information using the terminal device 200 to the content providing server 300. [

If the sensor information is managed separately from the image data, the sensor information is also transmitted to the content providing server 300.

At this time, the user information may be known through the MAC address, which is unique identification information of the terminal device 200, and user information such as the age, sex, and residence area of the user may be obtained through the MAC address.

The user information may be stored in the content providing server 300 and may be extracted by the content providing server 300 through the unique identification information of the terminal device 200. [

Thereafter, the terminal device 200 receives one or more recommendation information from the content providing server 300. [

The terminal device 200 of the present invention can be implemented in various forms. For example, the terminal device 200 described in the present specification may be a mobile terminal such as a smart phone, a tablet PC, a PDA (personal digital assistant), a portable multimedia player (PMP) Of course, fixed terminals such as smart TVs, desktop computers, etc. may be used.

In addition, the terminal device 200 of the present invention can not enumerate all of the variations of the portable device according to the convergence trend of the digital device, but a unit equivalent to the above- The beacon service app can be used as the terminal device 200 according to the present invention and any device can be used as the terminal device 200 according to the embodiment of the present invention.

The content providing server 300 manages the collected images, receives a service provision request from the terminal 300, provides information corresponding to the service, and provides schedule information required by the third service server 400 .

In particular, in the present invention, the content providing server 300 receives and stores at least one image data and sensor information corresponding to each of the at least one image data from the terminal device 200, May further receive and store user information from the terminal device 200. [

The content providing server 300 analyzes image and context information included in the image data based on the received at least one image data and extracts fact information based on the analyzed image analysis information and the received sensor information , And extracts the estimation information based on the analyzed context information.

At this time, the sensor information includes the position information, the date information, the weather information, and the like of the terminal device 200 at the time the image data is captured. The sensor information may be separately managed, .

The image analysis information is an analysis of various information included in images such as photographs, and may be a signboard of a shop included in the image, a trademark of clothes, a name of a specific building, and the like.

Context analysis information refers to contextual information displayed in an image, and is information obtained by analyzing the behavior of a specific person included in the image, the distance in the image, and the situation of the place.

Therefore, the fact information is information on facts corresponding to the images obtained through the sensor information and the image analysis information, and information about facts that can be left as data for the photographed place at the time the image was photographed.

On the other hand, the estimated information is information obtained by estimating the situation in the image obtained through the context analysis information, and may be estimated information about a specific situation that is estimated to have occurred at the photographed place at the time of capturing the image.

When the content providing server 300 extracts the fact information and the estimation information, at least one of the extracted estimation information, fact information, and user information and the corresponding image data may be mapped and stored.

And classifying the received at least one image data according to at least one criterion set based on at least one of the fact information and the estimation information for each of the image data to classify the classified reference or the at least one classified image data One or more pieces of information may be transmitted to one or more terminal devices or a third service server.

In other words, classifying at least one image data according to one or more criteria set based on at least one of the fact information and the presumption information may be based on the fact that the location, time, weather, Refers to grouping at least one image data received based on at least one of a surrounding situation, a background, and a landscape in a photograph included in the information.

Based on the group information, that is, the fact information and the estimated information of the classified image data, one or more recommendation information is extracted and provided to the terminal 200 of the user.

That is, if the classification criteria of the classified image data is a travel place and the travel place is 'Jeju Island', the user recognizes that the Jeju Island is being traveled, and can extract and provide a recommendation trip or a restaurant of Jeju Island as recommendation information have.

When extracting such recommendation information, it is possible to provide a recommended travel destination by further utilizing the estimated information, and the estimated information is 'people are walking on the grass on horseback'. If you do, you might be able to recommend a place to travel or a place to visit.

A recommendation list including at least one recommendation information to be extracted based on the fact information and the estimated information when the recommendation information is extracted and provided to a user, The recommendation information corresponding to the recommendation information can be excluded.

That is, if the classified travel destination is 'Jeju Island' and at least one image is analyzed, if there is a picture taken in the vicinity of 'Manjanggul', it is determined that 'Manjanggul' has already been visited by the user, Information can be excluded and the remaining recommendation information can be provided to the user.

In addition, after at least one image data is classified, that is, at least one image data is grouped and then one or more image data is classified according to a certain criterion, and classified according to a user's request or need, One or more image data can be provided to the user.

On the other hand, when transmitting at least one classified image data to the third service server 400, characteristic information for each user is obtained from at least one image data classified based on at least one of extracted fact information and estimated information And extract and extract the characteristic information to the third service server 400.

That is, it is possible to determine whether the user who has transmitted the at least one image data to the content providing server 300 based on at least one of the fact information and the estimation information, for example, The user can extract the taste such as enjoying a gastronomic trip and transmit the extracted characteristic information per user to the third service server 400.

On the other hand, at the time of classification, related image data are primarily classified according to a certain criterion on the basis of fact information, and then primary classified image data is classified into image data having certain characteristics based on the estimated information .

That is, the image data related to 'Jeju Island' is first classified into the image data based on the fact information, and then the congested photographs having many people based on the estimated information among the image data associated with the 'Jeju Island' .

The third service server 400 receives user-specific characteristic information from the content providing server 300 and collects and stores the received information for marketing or the like.

The content providing server 300 according to the embodiment of the present invention has a hardware configuration similar to that of a typical Web server or a network server. However, the software includes a program module implemented through a language such as C, C ++, Java, Visual Basic, Visual C, or the like.

On the other hand, the memory mounted on each device of the present invention stores information in the device. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in other embodiments, the memory may be a non-volatile memory unit. In one implementation, the storage device is a computer-readable medium. In various different implementations, the storage device may comprise, for example, a hard disk device, an optical disk device, or any other mass storage device.

Although the present specification and drawings describe exemplary device configurations, the functional operations and subject matter implementations described herein may be embodied in other types of digital electronic circuitry, or alternatively, of the structures disclosed herein and their structural equivalents May be embodied in computer software, firmware, or hardware, including, or in combination with, one or more of the foregoing. Implementations of the subject matter described herein may be embodied in one or more computer program products, i. E. One for computer program instructions encoded on a program storage medium of the type for < RTI ID = 0.0 & And can be implemented as a module as described above. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects the machine readable propagation type signal, or a combination of one or more of the foregoing.

The structure of the content providing system according to the embodiment of the present invention has been schematically described above.

Hereinafter, the configuration and operation method of the terminal device 200 according to the present invention will be described.

FIG. 2 is a block diagram showing the main configuration of the terminal device 200, and FIG. 3 is a flowchart for explaining an operation method of the terminal device 200 according to the embodiment of the present invention.

2, a terminal device 200 according to the present invention includes an input module 210, an output module 220, a storage module 230, a communication module 240, a GPS module 250, (260), and a control module (270).

The input module 210 receives various information such as numbers and character information, and transmits various signals to the control module 270 in connection with setting various functions and controlling functions of the terminal device 200. In addition, the input module 210 may include at least one of a keypad and a touchpad that generates an input signal according to a user's touch or operation. At this time, the input module 210 may be configured in the form of a touch panel (or a touch screen) together with the output module 220 to simultaneously perform input and display functions. The input module 210 may be any input device such as a keyboard, a keypad, a mouse, a joystick, and the like. In particular, the input module 210 according to the present invention senses input information input from a user and transmits the input information to the control module 270.

Particularly, the input module 210 of the terminal device 200 according to the present invention receives a user's image capturing or generation request, an image capturing request, and the like so that the control module 270 can capture or capture an image And can transmit the input request.

The output module 220 displays information on a series of operation states, operation results, and the like that occur during the performance of the function of the terminal device 200. In addition, the output module 220 can display a menu of the terminal device 200 and user data input by the user. The output module 220 may be a liquid crystal display (LCD), a thin film transistor LCD (TFT-LCD), a light emitting diode (LED), an organic light emitting diode (OLED) An organic light emitting diode (OLED), an organic light emitting diode (OLED), a retina display, a flexible display, and a three-dimensional display. In this case, when the output module 220 is configured as a touch screen, the output module 220 may perform some or all of the functions of the input module 210.

In particular, in the present invention, recommendation information received from the content providing server 300 through the output module 220 can be exposed to the user.

The storage module 230 is a device for storing data, and includes a main storage device and an auxiliary storage device, and stores an application program necessary for the functional operation of the terminal device 200. [ The storage module 230 may include a program area and a data area. Here, when the terminal device 200 activates each function in response to a user's request, the terminal device 200 executes the corresponding application programs under the control of the control module 270 to provide each function.

In particular, the storage module 230 according to the present invention includes an operating system for booting the terminal device 200, various application programs, user information matched to the terminal device 200, and a camera module 260 of the terminal device 200 Image data information captured or captured, one or more recommendation information received from the content providing server 300, and the like.

The communication module 240 is for transmitting / receiving data to / from the content providing server 300 through the communication network 100.

The communication module 240 includes RF transmitting means for up-converting and amplifying the frequency of the transmitted signal, RF receiving means for performing low-noise amplification on the received signal and down-converting the frequency, Data processing means, and the like. The communication module 240 may include at least one of a wireless communication module (not shown) and a wired communication module (not shown). The wireless communication module is configured to transmit and receive data according to a wireless communication method. When the terminal device 200 uses wireless communication, the wireless communication module uses any one of a wireless network communication module, a wireless LAN communication module, and a wireless fan communication module And transmit and receive data to and from the content providing server 300. [

Here, the communication module 240 uses a wireless communication method such as WLAN (Wireless LAN), Wi-Fi, Wibro, WiMAX, HSDPA (High Speed Downlink Packet Access) However, the present invention is not limited to this, and it is also possible to use Ethernet, xDSL (ADSL, VDSL), HFC (Hybrid Fiber Coaxial Cable), FTTC (Fiber To The Curb), FTTH May be used.

The sensor module 250 is a device for sensing and acquiring various information that can be sensed in the vicinity of the terminal device 200.

A GPS (Global Positioning System) is a typical example of such a sensor module 250. In the present invention, the GPS receives a GPS signal, and when the terminal device 200 photographs or generates image data of a photograph or the like, 200 to acquire the position where the photograph or the like is photographed.

In addition, the sensor information for extracting the fact information may include temporal information, weather information, and the like in addition to the positional information on which the image of the photograph or the like is photographed. The temporal information may include a timer installed in the terminal device 200, And the weather information can be obtained through a temperature sensor, an illuminance sensor or the like mounted on the terminal device 200. [

However, the weather or date information may not be acquired by the sensor module 250, but may be acquired in association with the weather or calendar application installed in the terminal device 200. [

The camera module 260 is a device for photographing a landscape, a portrait, etc. in an area where the terminal device 200 is located.

In the present invention, the user can take a photograph or the like through the camera module 260 of the terminal device 200 and generate image data.

The control module 270 may be an operating system (OS) and a process device for driving each configuration.

The control module 270 of the terminal device 200 controls the transmission of the signal received through the input module 210 to the content providing server 300 through the communication module 240 and the communication module 240 The control unit 220 controls the output module 220 to expose information received from the content providing server 300 through the output module 220 and store the information or data in the storage module 230.

In particular, in the present invention, the control module 270 may control the overall operation of the terminal device 200 according to the embodiment of the present invention.

Hereinafter, an operation process of the terminal device 200, which is generally controlled by the control module 270, will be described with reference to FIG.

The user captures an image such as a photograph through the camera module 260 of the terminal device 200 or captures an image stored in the content providing server 300 or another image storage server (S101).

At this time, if the user photographs a photograph through the camera module 260, the position information of the terminal 200 when the photograph is taken through the sensor module 250, the date and time when the photograph was taken, Sensor information such as weather information is acquired (S103).

Such sensor information may be managed separately from the image data, or may be included in the metadata of the image data.

The control module 270 of the terminal device 200 may control to transmit the captured or captured image data together with the sensor information and the user information to the content providing server 300 (S105).

After the control module 270 of the terminal device 200 transmits the information to the content providing server 300 through the communication module 240, the terminal device 200 transmits the information from the content providing server 300 to the communication module 240) (S107), and control the user to provide the one or more recommendation information through the output module 220. [0050] FIG.

The overall operation of the terminal device 200 according to the embodiment of the present invention can be controlled and executed by the control module 270 of the terminal device 200 as described above.

The configuration and operation method of the terminal device 200 according to the present invention have been described above.

Hereinafter, a configuration and an operation method of the content providing server 300 according to the present invention will be described.

FIG. 4 is a diagram illustrating a main configuration of a content providing server 300 according to the present invention, and FIGS. 5 to 7 are flowcharts for explaining an operation procedure of a content providing server 300 according to an embodiment of the present invention.

4, a content providing server 300 according to the present invention may include a communication module 310, a storage module 330, and a control module 350.

The communication module 310 is for communicating with the terminal 200. The communication network for performing the functions of the communication module 310 may be a wireless LAN (WLAN), a Wi-Fi, a WiBro, a WiMAX However, the present invention is not limited to this, but it is possible to use a wireless communication system such as Ethernet, xDSL (ADSL, VDSL), HFC (Hybrid Fiber Coaxial) Cable, Fiber to the Curb (FTTC), and Fiber To The Home (FTTH).

In particular, in the present invention, the communication module 310 can receive image data, sensor information, user information, and the like from the terminal device 200 through the communication network 100, and transmits the extracted one or more recommendation information to the terminal device 200).

The storage module 330 is a device for storing data, and includes a main storage device and an auxiliary storage device, and stores an application program required for the functional operation of the terminal device 200. [ The storage module 330 may include a program area and a data area. Here, when the terminal device 200 activates each function according to a request from the user, the terminal device 200 executes the corresponding application programs under the control of the control module 350 to provide each function.

In particular, in the present invention, the storage module 330 may classify image data according to one or more criteria set based on at least one of the fact information and the estimation information for each image data, and store the image data in the form of an image grouping DB 331 .

In order to provide one or more recommended information based on the fact information and the estimated information, the one or more recommendation information may be stored in the form of a recommendation information DB 333.

The control module 350 extracts one or more pieces of customized recommendation information that are personalized to the user based on the image data and the sensor information received through the communication module 310 and transmits the customized recommendation information to the terminal device 200 or the third service server 400 As shown in Fig.

The control module 350 controls the overall operation of the content providing server 300. The overall operation of the content providing server 300 controlled by the control module 250 will be described with reference to FIGS. .

5, the content providing server 300 receives image data, user information, and sensor information corresponding to the image data from the terminal device 200, and stores the received sensor information in the storage module 330 (S201).

The received sensor information may include at least one of position information, date information, and weather information of the terminal device 200 that generated or photographed the image data at the time when the image data was created or photographed, The sensor information including the position information may be acquired by the sensor module 250 of the terminal device 200.

Thereafter, the content providing server 300 extracts image analysis information by analyzing the characters or graphics included in the received image data, analyzes the context information indicating the status included in each of the image data (S203) And extracts the fact information based on the received information and the received sensor information (S205).

Then, in step S207, the estimation information is extracted based on the analyzed context information, and at least one of the estimated information and the fact information is mapped to the image data and stored in the storage module 330 in step S209.

Referring to FIG. 6, a process of providing recommendation information to the terminal 200 may include providing at least one image data according to one or more criteria set based on at least one of the fact information and the estimation information. I.e., grouped (S301).

At this time, the at least one image data may be firstly classified into related image data according to a certain reference based on the fact information, and then image data having a predetermined characteristic may be secondarily classified based on the estimated information.

The user's situation is analyzed based on the fact information and the estimation information from the classified image (S303). In other words, it is determined from the classified image, based on the fact information and the estimated information, Or a lot of photographs taken in one place may mean that the user is interested in such a place, so that the above information, that is, the user's situation, can be analyzed through the classified image data.

One or more recommendation information is extracted based on the analyzed user's situation (S305), and the extracted recommendation information may be transmitted to the terminal device 200 and provided to the user (S307).

At this time, the content providing server 300 generates a recommendation list including the extracted one or more recommendation information, and selects a recommendation list from among the recommended recommendation information included in the recommended recommendation list, Information can be extracted and provided to the user.

That is, it is highly likely that the place included in the photograph captured or photographed by the user is already traveled by the user, so it may be more helpful for the user to recommend the travel destination other than the travel destination, It is possible to provide the user with the travel destination information except for the place where it is determined that the travel destination has been visited.

Meanwhile, in the embodiment of the present invention, the content providing server 300 may provide at least one image data classified according to the same criterion to the terminal device 200 according to the request or need of the user.

Referring to FIG. 7, the content providing server 300 provides user-specific characteristic information to the third service server 400. At least one user provides image data stored in the storage module 330 (S401), extracts characteristic information for each user based on the fact information and estimated information from the classified image data (S403), and extracts characteristic information for each user based on the fact information and the estimated information, The extracted characteristic information may be transmitted to the third service server 400 (S405).

That is, the service provider that extracts the characteristic information of the favorite travel destination, the preference restaurant, etc. for each user from the classified image data and transmits it to the third service server 400 and manages the third service server 400, It can be utilized for marketing.

The configuration and operation method of the content providing server 300 according to the present invention have been described above.

Hereinafter, a process of analyzing image data according to an embodiment of the present invention and extracting recommendation information will be described.

8 to 11 are diagrams for explaining a process of extracting recommendation information according to an embodiment of the present invention.

8, at least one photograph (a) to (f) related to Jeju Island taken by at least one user while traveling on Jeju Island and sensor information corresponding to each photograph are transmitted to the contents providing server 300, The content providing server 300 may analyze the image analysis information by analyzing the characters or graphics included in the image data based on the respective image data, .

8, the location information commonly included in the six photographs included in the sensor information is 'Jeju Island', and the date and time information is 2015 August 3, and the weather information is assumed to be recorded as a clear weather of 33 degrees Celsius.

In addition, through the image analysis information, the information such as' Jeonbokjuk ',' Sobangsan ',' Sangsangjiri ',' Dolhulbang ',' Meat Noodle ',' Quot ;, " looking at food ", " looking at food ", and " looking at food "

(B), (c), (d), and (d), which photographs the six photographs based on the location information among the extracted fact information according to the criterion of 'Jeju Island' , (f), and (c) food photographs (a) and (e), and based on such information, recommendation information including a recommended travel destination and a recommended restaurant can be extracted and provided to the user.

FIG. 9 shows an example of the recommendation information as shown in FIG. 9. Referring to FIG. 9, a recommendation tour course is prepared based on the photograph classification standards (b), (c), (d), and (f) To the user.

(B), (c), (d), and (f) are likely to be places where the user has already traveled. Based on the remaining places, the user can provide a recommended tour course.

On the other hand, the content providing server 300 grasps the characteristic information of the user who likes to enjoy the leisure scenery and enjoys the gastronomic journey based on the extracted fact information and the estimated information, (400), the third service provider can utilize it for marketing.

10 through 11 illustrate another embodiment of the present invention. FIG. 10 shows image data uploaded by at least one user, and sensor information included in the photograph of FIG. 10 is different.

The content providing server 300 extracts common image analysis information called 'rock festival' from six photographs through image analysis, and extracts common image analysis information such as 'many people', 'crowded', 'many lights are shining on the stage' Context information.

The content providing server 300 extracts the fact information based on the image analysis information and the sensor information, and extracts the estimation information based on the context information.

(A) and (e), in which many lights are illuminated on the basis of the estimated information, are classified into the remaining 4 The photographs are classified separately.

Then, based on the classified information, one or more lock festival information that the user may be interested in may be extracted and provided to the user as shown in FIG.

Referring to FIG. 11, it is shown that a poster and a schedule of a rock festival of 2016 which the user is interested in based on the fact information and the estimation information of at least one image data analyzed by the grouping.

On the other hand, the content providing server 300 analyzes the at least one image data to determine whether the user wants to participate in a rock festival, And transmit it to the third service server 400.

In addition, among the six photographs classified in FIG. 10, (a) and (e), which are classified according to the same criterion as 'a photograph in which many lights are staged' have.

The process of analyzing image data and extracting recommendation information according to an embodiment of the present invention has been described above.

12 is a diagram illustrating an operating environment of an apparatus for providing recommended content in a recommended content providing system according to an embodiment of the present invention.

Figure 12 and the following discussion are intended to provide a brief, general description of a suitable computing environment in which the invention may be implemented. Although not required, the invention may be described in connection with computer-executable instructions, such as program modules, being executed by a computer system.

Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer executable instructions, associated data structures, and program modules illustrate examples of program code means for carrying out the acts of the invention disclosed herein.

10, an exemplary computing system embodying the present invention includes a processing unit 11, a system memory 12, and various system components including the system memory 12 connected to the processing unit 11 And a system bus 10 for communicating with the system.

The processing unit 11 may execute computer-executable instructions designed to implement the features of the present invention.

The system bus 10 may be any of several types of bus structures including a local bus, a peripheral bus, and a memory bus or memory controller using any of a variety of bus architectures.

The system memory 12 includes a ROM (Read Only Memory) 12a and a RAM (Random Access Memory) 12b. A basic input / output system (BIOS) 13a, containing the basic routines that help to transfer information between components within a computing system, such as during start-up, may generally be stored in ROM 12a.

The computing system may include storage means, for example, a hard disk drive 15 that reads information from, or writes information to, the hard disk, reads information from, or writes information to, And an optical disk drive 17 that reads information from, or writes information to, an optical disk such as, for example, a CD-ROM or other optical media .

The hard disk drive 15, the magnetic disk drive 16 and the optical disk drive 17 are connected by a hard disk drive interface 18, a magnetic disk drive-interface 19 and an optical drive interface 20, respectively, And is connected to the bus 10.

Further, the computing system may further include an external memory 21 as a storage means. The external memory 21 may be connected to the system bus 10 through an input /

The above-described drives and their associated computer-readable media readable and writable by the drives provide non-volatile storage of computer-executable instructions, data structures, program modules and other data. The exemplary environment described herein illustrates a hard disk 15, magnetic disk 16 and optical disk 17, but may also include magnetic cassettes, flash memory cards, DVDs, Bernoulli cartridges, RAMs Other types of computer readable media for storing data, including ROM, ROM, etc., may be used.

Includes one or more program modules including an operating system 13b, one or more application programs 13c, other program modules 13d, and program data 13c that are loaded and executed by the processing unit 11 The program code means may be stored in the hard disk 15, the magnetic disk 16, the optical disk 17, the ROM 12a or the RAM 12b.

In addition, the computing system may receive commands and information from a user through other input devices 22 such as a keyboard, a pointing device, a microphone, a joystick, a game pad, a scanner,

These input devices 22 may be connected to the processing unit 11 via an input / output interface 24 connected to the system bus 10. The input / output interface 24 may be, for example, a serial interface such as a serial port interface, a PS / 2 interface, a parallel port interface, a USB interface, an Institute of Electrical and Electronics Engineers (IEEE) 1394 interface (i.e., FireWire interface) Any of a variety of different interfaces can be represented logically, or even combinations of different interfaces can be represented logically.

In addition, the computing system to which the present invention is applied may further include a display device 26 such as a monitor or LCD or an audio device 27 such as a speaker or microphone, which may be connected via a video / audio interface 25 And is connected to the system bus 10.

For example, other peripheral output devices (not shown), such as speakers and printers, may also be connected to the computer system. The video / audio interface unit 25 may include a high definition multimedia interface (HDMI), a graphics device interface (GDI), or the like.

Further, the computing system embodying the invention is connectable to a network such as, for example, an office-wide or enterprise-wide computer network, a home network, an intranet, and / or the Internet. The computer system may exchange data with external sources such as, for example, a remote computer system, a remote application, and / or a remote database.

To this end, a computing system to which the present invention is applied includes a network interface 28 for receiving data from an external source and / or for transmitting data to an external source.

In the present invention, such a computing system is capable of transmitting and receiving information with a device located remotely via the network interface 28. For example, when the computing system refers to the terminal device 200, information can be exchanged with the content providing server 300 through the network interface 28. [

On the other hand, when the computing system refers to the content providing server 300, information can be exchanged with the terminal device 200 or the third service server 400 through the network interface 28. The network interface 28 may be represented by a logical combination of one or more software and / or hardware modules, such as, for example, a network interface card and a corresponding Network Driver Interface Specification ("NDIS") stack.

Likewise, the computer system receives data from an external source via the input / output interface 24 or transmits data to an external source. Output interface 24 may be coupled to a modem 23 (e.g., a standard modem, a cable modem, or a digital subscriber line (" DSL ") modem) Lt; / RTI > and / or transmit data to an external source.

Although FIG. 12 shows an operating environment suitable for the present invention, the principles of the present invention may be employed with any system capable of implementing the principles of the present invention, with appropriate modifications if necessary. The environment shown in Fig. 12 is merely illustrative and does not represent a small portion of a very diverse environment in which the principles of the present invention may be implemented.

In addition, various information generated when determining whether or not to enter the store of the present invention can be stored and accessed in any computer-readable medium related to the computing system. For example, a portion of these program modules and a portion of the associated program data may be stored in the system memory 12, such as in the operating system 13b, the application program 13c, the program module 13d, and / (13e).

Further, when a mass storage device such as a hard disk is connected to the computing system, such program modules and related program data may be stored in the mass storage device. In a networked environment, program modules associated with the present invention, or portions thereof, may be stored in a remote computer system connected via a modem 23 or network interface 25 of the input / output interface 24. The execution of such a module can be performed in a distributed environment as described above.

As described above, the present specification contains details of a number of specific implementations, but they should not be construed as being limitations on the scope of any invention or claimability, but rather on the particular embodiment of a particular invention But should be understood as an explanation of the features. Certain features described herein in the context of separate embodiments may be implemented in combination in a single embodiment.

Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Further, although the features may operate in a particular combination and may be initially described as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, Or a variant of a subcombination.

Likewise, although the operations are depicted in the drawings in a particular order, it should be understood that such operations must be performed in that particular order or sequential order shown to achieve the desired result, or that all illustrated operations should be performed. In certain cases, multitasking and parallel processing may be advantageous.

Also, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems will generally be integrated together into a single software product or packaged into multiple software products It should be understood.

Certain embodiments of the subject matter described herein have been described. Other embodiments are within the scope of the following claims. For example, the operations recited in the claims may be performed in a different order and still achieve desirable results. By way of example, the process illustrated in the accompanying drawings does not necessarily require that particular illustrated or sequential order to obtain the desired results. In certain implementations, multitasking and parallel processing may be advantageous.

The description sets forth the best modes of the present invention and provides examples for the purpose of illustrating the invention and enabling a person skilled in the art to make and use the invention.

The written description is not intended to limit the invention to the specific terminology presented. Thus, while the present invention has been described in detail with reference to the above examples, those skilled in the art will recognize that modifications, changes, and modifications can be made thereto without departing from the scope of the present invention.

Therefore, the scope of the present invention should not be limited by the described embodiments but should be defined by the claims.

The present invention relates to a method and apparatus for providing recommended contents. More particularly, the present invention relates to a method and apparatus for providing recommended contents by extracting fact information through sensor information and image analysis information of a photograph taken or captured by a user, extracting estimated information through context analysis information, And a method and an apparatus for classifying a plurality of image data according to a predetermined criterion on the basis of the fact information and the estimation information to provide customized recommendation contents specialized for the user.

According to the present invention, fact information is extracted using sensor information and data analysis information of a photograph taken or captured by a user, extracting estimation information using context information obtained by analyzing photographs, By extracting the recommendation information necessary for the user based on the information and providing the same to the user, it is possible to provide the customized content according to the change of the behavior of the user and the surrounding situation while managing the pictures more diversely and efficiently.

In particular, one or more predetermined criteria may be set based on the extracted fact information and the estimated information, and one or more pieces of information including recommendation information for classifying the pictures according to the set criteria and extracting them from the classified or classified pictures Providing the user with customized content tailored to the user's situation and providing the third service provider with one or more pieces of information that can be extracted from the classified criteria or the classified photos to provide an effective marketing strategy based on age, You can help them perform.

Therefore, the present invention can contribute to the development of the big data analysis industry and the storage platform service industry through the above-mentioned contents providing method, and it is possible to be industrially applicable since it is possible to carry out clearly on the market or business as well as being realistic .

100: communication network 200: terminal device 300: content providing server
400: Third service server

Claims (18)

Receiving and storing at least one piece of image data and sensor information corresponding to each piece of image data from at least one terminal device;
Extracting image analysis information by analyzing a character or graphic included in each of the image data based on the image data;
Analyzing contextual information indicating a situation contained in each of the image data based on the image data;
Extracting fact information on each of the image data based on the extracted image analysis information and the sensor information;
Extracting estimation information for each of the image data based on the analyzed context information;
Classifying the at least one image data according to at least one criterion set based on at least one of fact information and estimation information for each of the image data; And
Transmitting at least one information related to the at least one classified image data to the at least one terminal device or the third service server;
The content providing method comprising the steps of: The method according to claim 1,
Extracting at least one recommendation information based on the fact information and the estimation information of the classified image data;
The method of claim 1, further comprising: The method of claim 2, wherein the extracting of the recommendation information comprises:
Generating a recommendation list including at least one recommendation information based on the extracted fact information and the estimated information; And
Extracting recommendation information other than recommendation information corresponding to the image analysis information from at least one recommendation information included in the recommendation list;
The content providing method comprising the steps of: 2. The method of claim 1,
Extracting characteristic information for each user from the classified image data based on at least one of the fact information and the estimation information;
The method of claim 1, further comprising: The method according to claim 1,
Providing the at least one image data sorted according to the same criteria to the terminal device;
The method of claim 1, further comprising: 2. The method of claim 1,
Classifying the associated image data into a primary classification based on the fact information based on a predetermined criterion;
Classifying the primary classified image data into image data having a predetermined characteristic based on the estimated information;
The content providing method comprising the steps of: The method according to claim 1,
Mapping at least one of the fact information and the estimation information for each of the terminal devices to the image data and storing the information;
The method of claim 1, further comprising: The method of claim 1,
Wherein the terminal device includes at least one of location information, date information, and weather information of the terminal at the time when the terminal device generates the image data. A computer-readable recording medium on which a program for executing the method according to any one of claims 1 to 8 is recorded. 9. A computer program embodied on a computer readable recording medium which is embodied to perform the method recited in any one of claims 1 to 8. A communication module for transmitting and receiving data to and from at least one terminal device and a third service server in cooperation with a communication network;
A storage module for storing at least one image data received from the at least one terminal device and sensor information corresponding to each of the image data; And
Extracts image analysis information by analyzing a character or a graphic included in each of the image data based on the image data stored in the storage module and analyzes context information indicating a situation included in each of the image data based on the image data Extracting fact information based on the analyzed image analysis information and the sensor information, extracting estimation information based on the context information, and extracting at least one of the extracted fact information and recommendation information based on at least one of the extracted fact information and recommendation information, A control module for classifying image data and controlling one or more pieces of information associated with the at least one classified image data to be transmitted to the at least one terminal device or the third service server through the communication module;
And a content server. 12. The apparatus of claim 11, wherein the control module
And extracts one or more recommendation information based on the fact information and the estimation information of the classified image data. 13. The system of claim 12, wherein the control module
Generating a recommendation list including at least one recommendation information based on the extracted fact information and estimated information, and recommending remaining recommendation information excluding recommendation information corresponding to the image analysis information from at least one recommendation information included in the recommendation list And extracts the content. 12. The apparatus of claim 11, wherein the control module
And extracts characteristic information for each user from the classified image data based on at least one of the fact information and the estimation information. 12. The apparatus of claim 11, wherein the control module
And provides the at least one image data classified according to the same criteria to the terminal device. 12. The apparatus of claim 11, wherein the control module
And classifying the image data associated with each other according to a predetermined criterion on the basis of the fact information and classifying the image data classified in the first order into image data having a predetermined characteristic based on the estimated information. server. 12. The apparatus of claim 11, wherein the control module
Wherein at least one of the fact information and the estimation information for each of the terminal devices is mapped to the image data and is stored in the storage module. 12. The method of claim 11, wherein the sensor information
Wherein the content providing server includes at least one of location information, date information, and weather information of the terminal at the time when the terminal generates the image data.

Images