KR102582410B1 - System for providing augement reality based election campaign service - Google Patents

Abstract

An AR-based election campaign service provision system is provided, and a user terminal that transmits GPS information when filming election campaign promotional materials and outputs AR content corresponding to the GPS information, current time, and AR markers included in the election campaign promotional materials on the screen, and A database storage unit that maps and stores GPS information where election campaign promotional materials are installed, AR markers printed in election campaign promotional materials, and AR content by time, and detection that detects AR markers included in election campaign promotional materials when photographing election campaign promotional materials on a user terminal. It includes a campaign service providing server including an output unit that outputs AR content corresponding to the GPS information, current time, and AR marker of the user terminal on the screen of the user terminal.

Description

AR-based election campaign service provision system {SYSTEM FOR PROVIDING AUGEMENT REALITY BASED ELECTION CAMPAIGN SERVICE}

The present invention relates to an AR-based election campaign service provision system, and provides a platform that outputs different AR content for election campaign promotion depending on time, place, and user.

With the advent of the smart communication era, many election candidates are producing mobile-based election-related content for voters. Furthermore, this aspect has rapidly emerged as a new election campaign channel with infinite expansion potential by enabling digital communication between candidates and voters. Media used for political advertising include interpersonal media that directly exposes oneself in front of voters, organizational media that moves systematically for candidates, banners in outdoor spaces, exhibition media such as outdoor advertising, print media based on print, and TV media. , and is an internet medium that opens two-way communication. Voters are greatly influenced by the various media used during election campaigns. In the age of mass media, media such as television were the tools with the most decisive influence, but the emergence of the Internet made them more influential than newspapers or broadcasting in terms of the scope and speed of spread. is having a greater social and political impact.

At this time, a method of conducting an election publicity campaign using augmented reality was researched and developed. In relation to this, the prior art, Korea Registered Patent No. 10-1985640 (announced on June 4, 2019) and Korea Public Patent No. 2014- In No. 0015946 (released on February 7, 2014), election campaign data including election campaign audio data output through an election campaign display and candidate motion data that directs the motion of a 3D virtual model is generated and corresponds to the candidate. After extracting the 3D virtual model and loading it into the augmented reality environment, the 3D virtual model changes according to the candidate motion data and outputs audio according to the election campaign audio data through the display unit located on both sides. , In order to update multimedia content for election publicity campaigns to augmented reality in real time, a configuration is disclosed that displays objects recognized by augmented reality on the screen and streams pre-stored augmented reality content to a user terminal in real time.

However, in the former case, it does not provide information or pledges about the candidate through augmented reality, but merely displays the candidate campaigning in the front from the back. In the latter case, unless it is a presidential election, the same candidate is shown nationwide. Even though it is not displayed, the same content is mapped and stored to the same AR marker, so content for the same candidate is bound to be output, and if different candidates appear depending on the constituency or region, different AR markers are recreated and the content is stored one by one. It must be mapped. In addition, although the pledges of interest to each voter are different, because the pledges are presented in the same order, there are cases where voters vote without even reading all of the pledges. Accordingly, research and development of a platform that provides customized promises to users and provides different AR content depending on time and place when providing AR content based on GPS in places where election promotional materials and election banners are installed is required.

One embodiment of the present invention maps and stores the location of election campaign promotional materials, AR markers, and AR content to provide different AR content depending on location, time, and user, and stores the GPS and user information received from the user terminal. AR content mapped and stored to AR markers captured on the terminal is controlled to be output from the user terminal, and AR content already mapped and stored in response to the number of remaining voting days is output from the user terminal, so that the information is not always the same as election day approaches. It allows output of different AR content, and when interest data from user terminals is collected and clustered into at least one group, target content for each group with the order of pledges in the AR content changed is output from user terminals within the group, and election By examining the Beam Project in campaign promotional materials, we can provide an AR-based election campaign service provision system that can provide immersive media art in the evening hours. However, the technical challenge that this embodiment aims to achieve is not limited to the technical challenges described above, and other technical challenges may exist.

As a technical means for achieving the above-described technical problem, an embodiment of the present invention transmits GPS information when photographing election campaign promotional materials, and uses AR corresponding to GPS information, current time, and AR markers included in election campaign promotional materials. User terminal that outputs content on the screen and GPS information where election campaign promotional materials are installed, AR markers printed in election campaign promotional materials and database storage unit that maps and stores AR content by time, election campaign promotional materials when filming election campaign promotional materials on a user terminal It includes a campaign service providing server that includes a detection unit that detects the AR marker included in the user terminal, an output unit that outputs the GPS information of the user terminal, the current time, and AR content corresponding to the AR marker on the screen of the user terminal.

According to one of the above-described means for solving the problem of the present invention, the location, AR marker, and AR content of the election campaign promotional material are mapped and stored to provide different AR content according to location, time, and user, and the user terminal. It controls the AR content mapped and stored to the GPS received from and the AR marker taken from the user terminal to be output from the user terminal, and the already mapped and stored AR content is output from the user terminal in response to the number of voting days remaining, so that the same information is always provided. As the election day approaches, different AR content can be output, and when interest data from user terminals is collected and clustered into at least one group, target content for each group with the order of pledges in the AR content changed is displayed to user terminals within the group. It is possible to provide immersive media art in the evening by printing it out and using the beam projector on election campaign promotional materials.

1 is a diagram for explaining an AR-based election campaign service provision system according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a campaign service providing server included in the system of FIG. 1.
Figures 3 and 4 are diagrams for explaining an embodiment in which an AR-based election campaign service is implemented according to an embodiment of the present invention.
Figure 5 is an operation flowchart to explain a method of providing an AR-based election campaign service according to an embodiment of the present invention.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected," but also the case where it is "electrically connected" with another element in between. . In addition, when a part is said to "include" a certain component, this does not mean excluding other components unless specifically stated to the contrary, but may further include other components, and one or more other features. It should be understood that it does not exclude in advance the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

The terms "about", "substantially", etc. used throughout the specification are used to mean at or close to that value when manufacturing and material tolerances inherent in the stated meaning are presented, and are used to enhance the understanding of the present invention. Precise or absolute figures are used to assist in preventing unscrupulous infringers from taking unfair advantage of stated disclosures. The term “step of” or “step of” as used throughout the specification of the present invention does not mean “step for.”

In this specification, 'part' includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized using one piece of hardware. Meanwhile, '~ part' is not limited to software or hardware, and '~ part' may be configured to reside in an addressable storage medium or may be configured to reproduce one or more processors. Therefore, as an example, '~ part' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or may be further separated into additional components and 'parts'. Additionally, components and 'parts' may be implemented to regenerate one or more CPUs within a device or a secure multimedia card.

In this specification, some of the operations or functions described as being performed by a terminal, apparatus, or device may instead be performed on a server connected to the terminal, apparatus, or device. Likewise, some of the operations or functions described as being performed by the server may also be performed in a terminal, apparatus, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal mean mapping or matching the terminal's unique number or personal identification information, which is identifying data of the terminal. It can be interpreted as

In this specification, election campaign promotional materials may include the National Election Commission's official presidential election poster, general election poster, local election poster, and the National Election Commission's official candidate information sheet sent by mail. At this time, the objects that are linked only with the application are the National Election Commission's official presidential election poster, general election poster, local election poster, and the National Election Commission's official candidate information sheet sent by mail, and the objects that the application and beam projector are linked with are the central election commission. This may include large banners installed on official buildings or structures registered by the Management Committee.

Hereinafter, the present invention will be described in detail with reference to the attached drawings.

1 is a diagram for explaining an AR-based election campaign service provision system according to an embodiment of the present invention. Referring to FIG. 1, the AR-based election campaign service providing system 1 includes at least one user terminal 100, a campaign service providing server 300, at least one analysis terminal 400, and at least one beam projector ( 500). However, since the AR-based election campaign service providing system 1 of FIG. 1 is only an embodiment of the present invention, the present invention is not limited to FIG. 1.

At this time, each component of FIG. 1 is generally connected through a network (Network, 200). For example, as shown in FIG. 1, at least one user terminal 100 may be connected to the campaign service providing server 300 through the network 200. In addition, the campaign service providing server 300 may be connected to at least one user terminal 100, at least one analysis terminal 400, and at least one beam projector 500 through the network 200. Additionally, at least one analysis terminal 400 may be connected to the campaign service providing server 300 through the network 200. In addition, at least one beam projector 500 may be connected to at least one user terminal 100, a campaign service providing server 300, and at least one analysis terminal 400 through the network 200.

Here, the network refers to a connection structure that allows information exchange between each node, such as a plurality of terminals and servers. Examples of such networks include a local area network (LAN) and a wide area network (WAN). Wide Area Network, Internet (WWW: World Wide Web), wired and wireless data communication network, telephone network, wired and wireless television communication network, etc. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), 5th Generation Partnership Project (5GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), and Wi-Fi. , Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth network, NFC ( It includes, but is not limited to, Near-Field Communication (Near-Field Communication) network, satellite broadcasting network, analog broadcasting network, and DMB (Digital Multimedia Broadcasting) network.

In the following, the term at least one is defined as a term including singular and plural, and even if the term at least one does not exist, each component may exist in singular or plural, and may mean singular or plural. This should be self-explanatory. In addition, whether each component is provided in singular or plural form may be changed depending on the embodiment.

At least one user terminal 100 photographs an election campaign promotional material using an AR-based election campaign service-related web page, app page, program, or application, and uses an AR marker in the election campaign promotional material and GPS information corresponding to the current location. It may be a terminal that transmits to the campaign service providing server 300. At this time, the user terminal 100 may be a terminal that is clustered by the campaign service providing server 300 and belongs to a group through at least one social media, and is microtargeted by the campaign service providing server 300 to provide a news feed. It may be a terminal that outputs from at least one social media.

Here, at least one user terminal 100 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop equipped with a navigation system and a web browser, a desktop, a laptop, etc. At this time, at least one user terminal 100 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one user terminal 100 is, for example, a wireless communication device that guarantees portability and mobility, and includes navigation, personal communication system (PCS), global system for mobile communications (GSM), personal digital cellular (PDC), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) ) It may include all types of handheld-based wireless communication devices such as terminals, smartphones, smartpads, and tablet PCs.

The campaign service providing server 300 may be a server that provides an AR-based election campaign service web page, app page, program, or application. In addition, the campaign service providing server 300 may be a server that maps and stores GPS information where election campaign promotional materials are installed, AR markers printed in election campaign promotional materials, and AR content by time. In addition, the campaign service providing server 300 divides all voters into groups by clustering them using at least one characteristic, compares keywords corresponding to the characteristics of each group with words included in the pledge list, and determines the similarity between the keywords and the keywords. It may be a server that rearranges the priority of the pledge list in descending order and then presents pledges customized to the user. And, the campaign service providing server 300 is a server that performs microtargeting to extract and provide pledges with the highest similarity to keywords corresponding to the characteristics of each group through the news feed of at least one social media of the user terminal 100. It can be. Additionally, the campaign service providing server 300 may be a server that controls the on/off time and light intensity to irradiate light to election campaign promotional materials using at least one beam projector 500. To this end, the campaign service providing server 300 may be a server that collects GPS-based illuminance, sunrise time, and sunset time. Additionally, the campaign service providing server 300 may be a server that analyzes each candidate's pledge list using at least one type of evaluation index using at least one analysis terminal 400 and then collects and posts the results.

Here, the campaign service providing server 300 may be implemented as a computer that can connect to a remote server or terminal through a network. Here, the computer may include, for example, a laptop equipped with a navigation system and a web browser, a desktop, a laptop, etc.

At least one analysis terminal 400 may be an analyst's terminal that analyzes and evaluates each candidate's pledge list using at least one evaluation index using an AR-based election campaign service-related web page, app page, program, or application. . At this time, the analysis terminal 400 can evaluate the specificity and feasibility of each candidate's pledge, and at least one evaluation index is used to evaluate the pledge and the candidate, such as the SMART index, FINE index, DREAM index, and MANIFESTO index. Evaluation indicators can be used, but are not limited to this.

Here, at least one analysis terminal 400 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop equipped with a navigation system and a web browser, a desktop, a laptop, etc. At this time, at least one analysis terminal 400 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one analysis terminal 400 is, for example, a wireless communication device that ensures portability and mobility, and includes navigation, personal communication system (PCS), global system for mobile communications (GSM), personal digital cellular (PDC), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) ) It may include all types of handheld-based wireless communication devices such as terminals, smartphones, smartpads, and tablet PCs.

At least one beam projector 500 may be a device that receives on/off and light intensity control from the campaign service providing server 300 using an AR-based election campaign service-related web page, app page, program, or application. At this time, the beam projector 500 may be a device that can map and store the installed location (GPS) and which election campaign promotional material it is projecting based on the AR marker. In addition, the beam projector 500 is turned on from sunset to sunrise, and may be turned on during the day depending on the illuminance, and emits brighter light when the surroundings are bright from sunset to sunrise. When the surroundings are dark, it may be a device that adjusts the light intensity to a default value.

Here, at least one beam projector 500 may be implemented as a computer that can connect to a remote server or terminal through a network. Here, the computer may include, for example, a laptop equipped with a navigation system and a web browser, a desktop, a laptop, etc. At this time, at least one beam projector 500 may be implemented as a terminal that can connect to a remote server or terminal through a network. At least one beam projector 500 is, for example, a wireless communication device that ensures portability and mobility, and is used for navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) ) It may include all types of handheld-based wireless communication devices such as terminals, smartphones, smartpads, and tablet PCs.

Figure 2 is a block diagram for explaining the campaign service providing server included in the system of Figure 1, and Figures 3 and 4 illustrate an embodiment in which an AR-based election campaign service is implemented according to an embodiment of the present invention. This is a drawing for this purpose.

Referring to FIG. 2, the campaign service providing server 300 includes a database unit 310, a detection unit 320, an output unit 330, a control unit 340, a customized promise unit 350, and an advertisement request unit 360. ), a cloud management unit 370, a date output unit 380, and a commitment analysis unit 390.

The campaign service providing server 300 or another server (not shown) operating in conjunction with an embodiment of the present invention includes at least one user terminal 100, at least one analysis terminal 400, and at least one beam projector. When transmitting an AR-based election campaign service application, program, app page, web page, etc. to 500, at least one user terminal 100, at least one analysis terminal 400, and at least one beam projector 500 You can install or open AR-based election campaign service applications, programs, app pages, web pages, etc. Additionally, a service program may be run on at least one user terminal 100, at least one analysis terminal 400, and at least one beam projector 500 using a script executed in a web browser. Here, a web browser is a program that allows the use of web (WWW: World Wide Web) services and refers to a program that receives and displays hypertext written in HTML (Hyper Text Mark-up Language), for example, Netscape. , Explorer, Chrome, etc. Additionally, an application refers to an application on a terminal and includes, for example, an app running on a mobile terminal (smartphone).

Referring to FIG. 2, the database unit 310 can map and store GPS information where election campaign promotional materials are installed, AR markers printed in election campaign promotional materials, and AR content by time. AR content according to an embodiment of the present invention consists of AR content for each group and AR content for each time, as shown in (b) of FIG. 3. For example, assuming that candidate A's AR content exists, there are N pieces of AR content for voter group 1, group 2,..., and group N. At this time, AR content may differ from one another, for example, by first presenting pledges that are most similar to the characteristics of each group. For example, if Voter Group 1 has the characteristic of considering the abolition of irregular workers as the top priority, for example, the 3rd pledge (pledge related to the abolition of irregular workers) from Candidate A's list of pledges related to the abolition of irregular workers is placed first. It is presented. In addition, hourly AR content continues to be polled as the voting day approaches. As the voting day approaches, the battle between candidates intensifies and public opinion changes accordingly, but providing the same AR content is not very effective in election campaigns. Accordingly, assuming that the election campaign period is 10 days, the AR content to be displayed on D-10 and the AR content to be displayed on D-1 can be set and prepared differently and made into a database. At this time, as the voting day approaches, there are many cases where the opponent's corruption is revealed and slandered, so AR content can be created in real time without saving it in advance and rendered in real time so that the AR content is streamed on that date.

The detection unit 320 may detect the AR marker included in the election campaign promotional material when the user terminal 100 photographs the election campaign promotional material. The user terminal 100 may transmit GPS information when photographing an election campaign promotional material and output AR content corresponding to the GPS information, current time, and AR marker included in the election campaign promotional material on the screen. At this time, a marker-based method is used to visualize content such as 3D models, single images, and alpha images by tracking predefined images or patterns, and a target image to be tracked by extracting color, object, or background characteristics from the original image. It can be distinguished by the markerless method used, but both methods have something in common in tracking objects and use the term marker, so the AR marker of the present invention is defined as including both methods.

Because AR markers directly affect the performance of object recognition and tracking, which are the main functions of augmented reality, managing AR markers is an important element in implementing an augmented reality system and maintaining performance. When using natural images as AR markers, AR marker recognition and tracking performance varies depending on the case, such as when the distribution of objects in the image is concentrated in a specific area, when it contains repetitive patterns such as texture, or when part of the image is lost due to lighting, etc. There may be a difference. Therefore, in one embodiment of the present invention, a method can be used to evaluate markers by reflecting the distribution of objects in election campaign promotional materials (people, objects, nature, etc. included in the shooting screen) and whether or not they contain repetitive patterns through feature point analysis. there is.

First, if the object in the captured screen is not evenly distributed throughout the image but is concentrated in a portion, the AR marker may be vulnerable to occlusion. Therefore, statistical characteristics of the location of feature points are used to determine whether the object is evenly distributed within the marker image. Available. In addition, the problem of finding a repeating pattern is that the distribution of the SIFT (Scale Invariant Feature Transform) descriptor of a general image can be approximated by a Gaussian distribution, so based on this, the similarity between the histogram and the Gaussian distribution for the feature point descriptor of the AR marker is measured. You can measure whether or not a repeating pattern is included.

First, feature points are detected using the SIFT technique. SIFT can extract feature points that are robust to changes in size, rotation, and illumination through the processes of detecting scale space poles, adjusting feature points, assigning direction information, and generating feature point descriptors. For example, a feature point descriptor may divide 16×16 pixels around a feature point into 16 4×4 blocks, obtain a direction histogram in 8 directions for each block, and express it as a 128-dimensional vector. The more feature points with various descriptor vectors, the easier it is to track or recognize objects in an image, and since natural images usually contain a variety of feature points, they can show excellent augmented reality performance when used as AR markers. However, in images containing objects containing artificial and repetitive patterns, such as buildings, the descriptor vectors representing feature points are the same or have very similar values, resulting in misregistration problems during the tracking or recognition process. Since an image containing multiple such descriptor vectors is inappropriate to use as a marker, its use may be determined based on the redundancy of the descriptor vectors.

In addition, by calculating the statistical characteristics, that is, variance, of the location coordinates of feature points that have gone through the feature point adjustment step in the SIFT technique, it is possible to determine whether feature points are evenly spread throughout the image. If the variance is low, respond to AR markers. Since objects are concentrated in some areas, most feature points may be lost even with slight occlusion, and augmented reality performance may deteriorate. Accordingly, a configuration for evaluating markers according to feature point distribution can be further implemented. Among the statistical characteristics of the location coordinates of feature points extracted by the SIFT technique, variance can be used to determine the effects of partial occlusion and noise on the marker image and use it as a marker evaluation factor. If objects are not evenly distributed throughout the image but are concentrated in some areas, the feature point coordinates have a low dispersion value, which means that there is a high possibility that many objects and feature points will be lost due to occlusion in the x or y direction, and at the same time, it is vulnerable to occlusion. Since it is an AR marker, it is best not to select it. The converse also holds true.

If a repetitive pattern exists, the feature points extracted by the SIFT technique, especially each component of the descriptor vector, overlap or have similar values. To check this, each descriptor vector must be compared, but the process of directly comparing 128-dimensional vectors is very inefficient. Accordingly, the effect of reducing the dimension can be achieved by obtaining the average descriptor vector within the captured screen and using the distance between each descriptor vector and the average descriptor vector. Looking at the histograms generated for various images, including repeating pattern images, we take advantage of the fact that while general natural images show a shape close to a Gaussian distribution, repeating pattern images appear in a non-standardized shape. This is because a general image has an abundance of feature points made up of various elements, so the distance from the average descriptor vector shows a Gaussian distribution, whereas in a repeating pattern image, many components of the descriptor vector overlap, so the distance from the average descriptor vector is distributed to a specific component. Because it is focused. Accordingly, the repetitive pattern marker image can be evaluated using the difference from the Gaussian distribution with the histogram distribution, mean, and variance as parameters.

The output unit 330 may output AR content corresponding to the GPS information, current time, and AR marker of the user terminal 100 on the screen of the user terminal 100. When an AR marker is found using the above-described method, AR content previously mapped and stored to the AR marker can be output from the user terminal 100. At this time, the reason for receiving GPS information is that, except for presidential elections, candidates are not the same in all regions. Even though there are different candidates for each region, if the AR marker is the same, AR content for the same candidate will be output unless GPS is used. In one embodiment of the present invention, when each candidate is different by region or when each candidate's election campaign promotional materials are located in different regions, for example, candidate A's election campaign promotional materials are located in Gangnam, and candidate B's election campaign promotional materials are located in Gangnam. If candidate C's election campaign promotional materials exist in Jongno and Nonhyeon, respectively, GPS can be extracted to determine which candidate's AR content should be output. Of course, you can also change the AR marker itself. For example, before the voting day, there are cases where the National Election Commission posts election campaign promotional materials for candidates 1 to N for each landmark in the same place. In this case, the AR markers themselves are produced differently and each landmark is posted in the same place. It can also be implemented to distinguish candidates.

The control unit 340 collects weather information corresponding to the GPS information where the election campaign promotional materials are located, and when the illuminance according to the current time among the weather information is less than the preset illuminance, drives the beam projector 500, and operates the illuminance according to the current time. The intensity of light of the beam projector 500 can be controlled to be inversely proportional to . The beam projector 500 can extract the sunset time and sunrise time based on GPS information where the election campaign promotional material is located, and irradiate light to the election campaign promotional material from sunset time to sunrise time. That is, the basic setting turns on the beam projector 500 from sunset to sunrise. In this case, if it rains or snows even during the day, or if the illumination is low due to clouds, the beam is turned on from sunrise to sunset. The projector 500 can be turned on. At this time, the illuminance for each GPS can be obtained based on GPS using the weather information API. Here, during the daytime, if the surroundings are bright, brightly, if the surroundings are bright at night, the settings will be brighter, and if the surroundings are dark during the night, the settings will be set to normal, etc., depending on the brightness of the surroundings, whether it is day or night, and depending on the illuminance. Turn on and off and light intensity can be adjusted.

The customized pledge unit 350 performs clustering to group all voters into at least one feature, and uses at least one feature extracted from at least one social media of the user terminal 100. Searches the group of the user terminal 100, measures the similarity between keywords corresponding to at least one characteristic clustering the group to which the user terminal 100 belongs, and the candidate's pledge list in the election campaign promotional material, and pledges with high similarity By arranging them in priority order, the display order of pledges in the AR content can be changed, and the changed AR content can be transmitted to the user terminal 100.

At this time, microtargeting is a method of maximizing the effectiveness of marketing activities by extracting a population and analyzing the actions and reactions of each micro unit, such as interests or personality, within that population to derive organic relationships and maximize the effectiveness of marketing activities. Also called targeting. In addition, because people disclose their information as profiles within social media, for example, Facebook, Twitter, Instagram, etc., basic access is possible and specific information can be accessed based on region, age, gender, interests, etc. This is a method of exposing advertisements only to targets with a profile. For example, there may be hundreds of thousands to millions of people with the characteristic of being a person who likes exercise, but if you add the characteristic of investing in stocks, the number increases to thousands to hundreds of thousands. It becomes narrow (Narrow Down). In addition, if more features are added, it becomes possible to target specific groups and send news feeds.

For example, a single-person household with characteristics such as animal protection (characteristic 1), preference for exercise (characteristic 2), preference for stock investment (characteristic 3), people in their 30s and 40s (characteristic 4), and opposition to the single tax (characteristic 5). If you want to promote Candidate A's pledge number 4 (pledge to support single-person households) by micro-targeting, set the characteristics of a group with the above-mentioned characteristics and send it to the social media of users in the group as NewsFeed. will be. In this case, candidate A's pledge number 4 is included in the newsfeed not only for user B but also for all user groups with characteristics 1 to 5. The method for creating each group is based on Euclidean distance-based similarity. In other words, it is based on the distance between users. The user is viewed as one location in many multidimensional spaces consisting of a coordinate system with preference values, and the Euclidean distance d3 between the two user locations is calculated. In other words, if the Euclidean distance has a large value, it means that the distance between users is long, so the similarity between users is low, so the value itself is not considered a meaningful similarity measure.

Alternatively, the K-means algorithm can be used. The K-means algorithm is an algorithm that groups given data into k clusters and operates by minimizing the variance of the distance difference between each cluster.

At this time, the goal of the algorithm is to find Si that minimizes the V value. The algorithm first starts by randomly setting the initial μi. Then repeat the next two steps. Cluster setting finds and assigns the closest cluster to each point. Reset μi to the average value of the points in each cluster. If the cluster does not change, stop the iteration.

Once 1 to N groups have been created through clustering, it is now necessary to identify the characteristics that represent each group. At this time, not only one feature is possible, but multiple features are also possible. Keywords can be extracted using word cloud, and the characteristics with which each group was created can be used as keywords. For example, continuing the example above, animal protection (feature 1), preference for exercise (feature 2), preference for stock investment (feature 3), people in their 30s and 40s (feature 4), opposition to single tax (feature 5) In this case, there are a total of 5 features, and the keywords could be [animal protection, exercise, stock investment, people in their 30s and 40s, and opposition to the single tax]. At this time, keywords corresponding to each feature are compared with words included in each candidate's pledge list. For example, assuming that Candidate A's 1st pledge is environmental protection, 2nd pledge is the abolition of irregular workers, 3rd pledge is to promote a single-person household policy, and 4th pledge is to raise the stock transfer tax standard, then 1st pledge [Environment] Protection] is similar to [Animal Protection (Feature 1)], Pledge 3, [Promotion of Single-Person Household Policy], is similar to [Opposition to Single Tax (Feature 5)], and Pledge 4, [Increasing the Stock Transfer Tax Standard] ] can correspond to [Stock Investment (Feature 3)]. Accordingly, the news feed provided to the above-mentioned group may be provided focusing on pledges 1, 3, and 4, and AR content is also rendered in real time by placing pledges 1, 3, and 4 first. It could be.

In order to determine the similarity between each word, the process of [NLP (Natural Language Processing) -> Stop Word Processing -> Word Vectorization] is performed. At this time, since NLP is a known technology, detailed explanation will be omitted. Stopwords are words that commonly appear frequently in any document within the set when a set of documents is given, especially the nature and relationship of the document set. There are words that appear frequently due to the function they have in a sentence. These words are called stopwords. Most of these stop words are words that do not have much meaning in themselves, but serve an important grammatical function. In the case of a model that simply uses high-frequency words as specific values for the document, specific values for stop words that are difficult to see as reflecting the actual meaning may be greatly highlighted, diluting the important characteristics of the actual document. When building a frequency-based model, it is common to exclude stop words from document features.

The Word2Vec model is a method of efficiently estimating the meaning of words by converting words into vectors of tens to hundreds of dimensions, enabling dramatic improvements in precision in the field of natural language processing. The Word2Vec model is based on an artificial neural network and starts from the premise that words in the same context have close meanings. The Word2Vec model learns through text documents and trains the artificial neural network on other words that appear together with one word in a sentence as related words. Because related words are more likely to appear nearby in a document, during repeated learning, two words with similar neighboring words are placed in a nearby vector space.

The Word2Vec model simply learns whether there is the same information before or after a word. Therefore, learning very abstract verbs or adjectives may be more difficult than learning nouns. However, by looking at countless amounts of data, it is possible to learn the semantic relationships between verbs to some extent by identifying the regularity of what objects the verbs have. The Word2Vec model is not a deep neural network (DNN). It is an artificial neural network consisting of one hidden layer to which no activation function is applied and an output layer to which the softmax function is applied. Therefore, a big advantage is that the learning speed is much faster than a typical deep neural network, so even very large data can be easily learned. The Word2Vec model's algorithm internally learns sentences using two neural network models, CBOW (Continuous Bag Of Words), which predicts words with a single context, and SG (Skip-Gram), which predicts context with words, to find words with similar meanings. Express them in a nearby vector space.

For example, when w(t) is given as an input word, SG trains a neural network with the goal of making predictions for a certain number of other words forward and backward according to the window size specified based on the input word. OUTPUT is a weight calculated to predict possible nearby words w(t-2), w(t-1), w(t+1), and w(t+2) based on the input word w(t). As a value, the weight values become vector values representing w(t). The Word2Vec model is given an input word and learns to maximize the result using the softmax function, which is the conditional probability of the output word. Equation 2 is the softmax function of the Word2Vec model.

P(W0|W1) is the conditional probability that W0 will appear as the output word given the input word W1. Accordingly, in the Word2Vec model, the closer the distribution of surrounding words in a learning document is, the more similar the calculated vector values are, and words with similar calculated vector values are considered similar. Of course, corpus can also be used in vector models.

In summary, the customized pledge unit 350 can use microtargeting when grouping all voters by at least one characteristic, and the advertisement request unit 360 can use at least one feature of the user terminal 100. You can request targeted advertising to send a list of pledges with the pledge display order changed to the NewsFeed of social media.

The cloud management unit 370 can control GPS information where election campaign promotional materials are installed, AR markers printed in election campaign promotional materials, and hourly AR content to be distributed and stored in the augmented reality cloud and receive real-time streaming from the user terminal 100. there is. At this time, the augmented reality cloud may be a cloud that performs real-time rendering and streaming based on edge computing. As described above, if all groups are set and each AR content is prepared in advance, real-time rendering is not necessary, but if new users come in and more groups are needed, or even within one group, different features are present, that is, feature 1 in group A. They have feature 5, but user B considers feature 1 more important, and user C considers feature 2 more important. If the order of presentation of pledges is the same for each group, it will only present group-customized pledges, but user-customized pledges. Promises cannot be made. Accordingly, pledges appropriate for the group are extracted through grouping, but the order of pledge presentation may be provided differently depending on the user, and real-time rendering may be necessary accordingly. For example, if the AR content was edited and rendered in the following order: Pledge 1 - Pledge 2 - Pledge 3 - Pledge 4 - Pledge 5, change the order to Pledge 2 - Pledge 1 - Pledge 4 - Pledge 5 - Pledge 3, etc. Real-time rendering can also be performed by deleting, etc.

At this time, edge computing is a technology that makes it possible to process a large amount of data through a lightweight user terminal 100 by utilizing computing resources located close to the user terminal 100. Each edge server shares content and is capable of real-time response to user requests and distributed computing, distributing network traffic and enabling fast response services, effectively reducing cloud data transmission and bottlenecks. there is. In addition, data security can be improved. In the centralized network structure, all data is transmitted to the cloud center, while in the distributed edge computing method, sensitive information and data are processed directly on the edge server, which increases the possibility of multiple users being threatened at the same time. can be lowered, only necessary information can be encrypted and processed, and even if the central server is down, service can be provided from the edge server, ensuring network stability. Due to the short-term nature of the election campaign, there may be a large crowd or delay, so in one embodiment of the present invention, edge computing is used to solve this problem. Of course, this does not rule out using a general cloud platform.

For example, when there is a request for AR content from the user terminal 100, the cloud management unit 370 allocates a session and continuously receives sensing data. The sensing data configuration receives the user's camera image information and IMU (Inertial Measurement Unit) data, and the input image is converted into an image and delivered to the Visual-SLAM module. The user's location and FOV (Field-Of-View) are extracted from the input image and IMU data. When the extracted data obtains a depth map through the 3D Reconstruction module, all data for rendering is prepared. Based on the current user's location and FOV data and spatial information, the cloud management unit 370 performs rendering and encoding simultaneously with the AR content to be served. It can be configured to be muxed (multiplexed) and transmitted on a frame-by-frame basis for a quick response to the user terminal 100. The cloud management unit 370 includes information about delay time and content information processed by the edge server in order to transmit accurate data to the end user, and may include metadata of rendering data.

The date-specific output unit 380 stores the voting date of the election, calculates the remaining voting days until the voting date based on the current date, and stores different AR contents corresponding to the remaining voting days by date. AR content by date is provided to the user. It can be output from the terminal 100.

The pledge analysis unit 390 may analyze the candidate's pledges in the election campaign promotional material using at least one evaluation index and upload and post the results of the analysis based on specificity and feasibility from the analysis terminal 400. At least one evaluation indicator is SMART (Specific, Measurable, Achievable, Relevant, Timed) indicator, SELF (Sustainability, Self-Government Capacity Strengthening) (Empowerment, Locality, Feedback) indicators, FINE (Feasibility, Interaction, Efficiency) indicators, DREAM (Vision (Dream), Affiliation, means of Indicators (Means, Responsiveness, Efficiency), MANIFESTO (Specific, Measurable, Aimed, National, Identical, Financing) (Financial), Timely, Efficient, Open, Consistent, Integral, Tolerable, Inventive, Do not spread false information (Zero False) Report), delivery ability (Enunciate), No Bragging) indicators, etc., but are not limited to these. It can be important not only to present a pledge, but also how specific and feasible it is. Accordingly, the analysis results according to the evaluation method of each indicator are uploaded together, and the analysis results of how likely each user is to implement the desired pledge are viewed together, allowing a more detailed analysis of the candidates.

Hereinafter, the operation process according to the configuration of the campaign service providing server of FIG. 2 described above will be described in detail using FIGS. 3 and 4 as an example. However, it will be apparent that the embodiment is only one of various embodiments of the present invention and is not limited thereto.

Referring to Figure 3, (a) the campaign service providing server 300 stores [candidate-GPS-AR marker-AR content] to build a database. At this time, (b) the campaign service providing server 300 may store AR content by group and AR content by time in order to provide each user with different AR content according to time, place, and user. And, (c) the campaign service provision server 300 builds a cloud platform to provide AR content to each user terminal 100 without interruption so that each AR content can be streamed to each user terminal 100 without delay. and (d) the campaign service providing server 300 adjusts the on/off and light intensity of the beam projector 500 so that the election campaign promotional material can be provided as immersive art. During the day, the light intensity is turned on only when the illuminance is below the standard value, and the light intensity is adjusted so that there is a large difference with the surrounding illuminance. At night, the light intensity is turned on from sunset to sunrise, and the light intensity can be adjusted so that there is not much difference with the surrounding illuminance. there is. Of course, the light intensity can be adjusted so that it differs greatly from the surrounding illumination.

Referring to (a) of Figure 4, for micro-targeting, all voters are divided into each group, the candidates' pledges are compared with the characteristics of each group, the pledges are rearranged in order of high similarity, and then provided as a news feed as shown in (b). It can be done, and it can also provide AR content with rearranged promises. At this time, AR content is not limited to a list of pledges, but can also include the candidate's growth or facial expression, the candidate's history, or criminal information, etc.

Matters that are not explained about the method of providing AR-based election campaign services in FIGS. 2 to 4 are the same as those explained about the method of providing AR-based election campaign services through FIG. 1, or can be easily inferred from the content explained. Therefore, the following description will be omitted.

FIG. 5 is a diagram illustrating a process in which data is transmitted and received between components included in the AR-based election campaign service provision system of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of the process of transmitting and receiving data between each component will be described with reference to FIG. 5, but the present application is not limited to this embodiment, and the process shown in FIG. 5 according to the various embodiments described above It is obvious to those skilled in the art that the process of transmitting and receiving data can be changed.

Referring to Figure 5, the campaign service providing server maps and stores GPS information where election campaign promotional materials are installed, AR markers printed in the election campaign promotional materials, and AR content by time (S5100), and photographs election campaign promotional materials on the user terminal. In this case, the AR marker included in the election campaign promotional materials is detected (S5200).

Additionally, the campaign service providing server outputs AR content corresponding to the GPS information, current time, and AR marker of the user terminal on the screen of the user terminal (S5300).

The sequence between the above-described steps (S5100 to S5300) is only an example and is not limited thereto. That is, the order between the above-described steps (S5100 to S5300) may change, and some of the steps may be executed simultaneously or deleted.

Matters that are not explained about the method of providing AR-based election campaign services in FIG. 5 are the same as or can be easily inferred from the contents explained about the method of providing AR-based election campaign services through FIGS. 1 to 4. Therefore, the following description will be omitted.

The method of providing an AR-based election campaign service according to an embodiment described with reference to FIG. 5 can also be implemented in the form of a recording medium containing instructions executable by a computer, such as an application or program module executed by a computer. . Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and non-volatile media, removable and non-removable media. Additionally, computer-readable media may include all computer storage media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data.

The method of providing an AR-based election campaign service according to an embodiment of the present invention described above can be executed by an application installed by default on the terminal (this may include programs included in the platform or operating system, etc., installed by default on the terminal). It may also be executed by an application (i.e., program) installed by the user directly on the master terminal through an application providing server such as an application store server, an application, or a web server related to the service. In this sense, the method of providing an AR-based election campaign service according to an embodiment of the present invention described above is implemented as an application (i.e., a program) installed by default in the terminal or directly installed by the user and can be read by a computer such as the terminal. It can be recorded on any recording medium.

The description of the present invention described above is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. will be. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as unitary may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

Claims (10)

A user terminal that transmits GPS information when photographing an election campaign promotional material and outputs AR content corresponding to the GPS information, current time, and AR marker included in the election campaign promotional material on the screen; and
A database unit that maps and stores GPS information where the election campaign promotional materials are installed, AR markers printed in the election campaign promotional materials, and AR content by time, and an AR marker included in the election campaign promotional materials when photographing the election campaign promotional materials on the user terminal. A detection unit that detects, GPS information of the user terminal, an output unit that outputs AR content corresponding to the current time and AR marker on the screen of the user terminal, GPS information where the election campaign promotional materials are installed, and printing in the election campaign promotional materials. A campaign service providing server including a cloud management unit that distributes and stores AR markers and hourly AR content in an augmented reality cloud and controls real-time streaming from the user terminal.
The augmented reality cloud is,
To reduce cloud data transmission volume and bottlenecks, real-time rendering and streaming are performed based on edge computing.
The cloud management department,
If there is a request for AR content from the user terminal, a session is assigned, sensing data including the user's camera image information and IMU (Inertial Measurement Unit) data is transmitted, and FOV (Field-Of-View) extracted from the sensing data Based on the current user's location and spatial information, it performs encoding at the same time as rendering with the AR content to be served, configures it to mux and transmit on a frame-by-frame basis for a quick response to the user terminal, and transmits accurate data to the end user. In order to do this, an AR-based election campaign service provision system is characterized by managing metadata of information about delay time, content information, and rendering data processed by the edge server.
According to claim 1,
The AR-based election campaign service provision system is,
A beam projector that extracts sunset time and sunrise time based on GPS information where the election campaign promotional material is located, and irradiates light to the election campaign promotional material from the sunset time to the sunrise time;
An AR-based election campaign service provision system further comprising:
According to claim 2,
The campaign service providing server is,
Collect weather information corresponding to GPS information where the election campaign promotional material is located, drive the beam projector when the illuminance according to the current time among the weather information is less than a preset illuminance, and operate the beam projector in inverse proportion to the illuminance according to the current time. A control unit that controls the intensity of light;
An AR-based election campaign service provision system further comprising:
According to claim 1,
The campaign service providing server is,
Clustering is performed to group all voters into at least one feature, and the group of the user terminal is searched using at least one feature extracted from at least one social media of the user terminal, Measure the similarity between the keyword corresponding to the at least one feature that clusters the group to which the user terminal belongs and the candidate's pledge list in the election campaign promotional material, arrange the pledges in order of priority in order of high similarity, and create the AR content. a customized pledge unit that changes the display order of my pledges and transmits the changed AR content to the user terminal;
An AR-based election campaign service provision system further comprising:
According to claim 4,
The customized contract section is,
An AR-based election campaign service providing system that uses microtargeting when grouping all voters by at least one characteristic.
According to claim 5,
The campaign service providing server is,
An advertisement request unit that requests a targeted advertisement to transmit a list of pledges in which the pledge display order has been changed to a NewsFeed of at least one social media of the user terminal;
An AR-based election campaign service provision system further comprising:
delete delete According to claim 1,
The campaign service providing server is,
Stores the voting date of the election, calculates the remaining voting days until the voting date based on the current date, stores different AR contents corresponding to the remaining voting days by date, and prints by date so that the AR content by date is output from the user terminal. wealth;
An AR-based election campaign service provision system further comprising:
According to claim 1,
The campaign service providing server is,
A pledge analysis unit that analyzes the candidate's pledges in the election campaign promotional material using at least one evaluation index and uploads and posts the results of the analysis based on specificity and feasibility from an analysis terminal;
An AR-based election campaign service provision system further comprising:

Images