KR101741584B1 - Mobile twrminal and screen configuration controlling method thereof - Google Patents

Abstract

The present invention relates to a mobile terminal having a virtual memo function and a method for managing the virtual memo. When a user creates a memo at a specific space and registers the virtual memo, the virtual memo is displayed So that the user can easily write and check the memo regardless of the place and time.

Description

TECHNICAL FIELD [0001] The present invention relates to a mobile terminal having a virtual memo function and a virtual memo management method therefor.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to augmented reality of a mobile terminal, and more particularly, to a mobile terminal capable of creating and displaying a virtual memo at a specific space, and a method of managing the virtual memo.

The mobile terminal may be configured to perform various functions. Examples of such various functions include a data and voice communication function, a function of photographing a video or a moving image through a camera, a voice storage function, a music file playback function through a speaker system, and an image or video display function. Some mobile terminals include additional functions to execute games, and some other mobile terminals are also implemented as multimedia devices. Moreover, recent mobile terminals can receive a broadcast or multicast signal to view a video or television program.

Further, efforts for supporting and increasing the functions of the mobile terminal continue. Such efforts include not only changes and improvements in structural components that form the mobile terminal, but also improvements in software or hardware. Among them, the touch function of the mobile terminal allows a user who is unfamiliar with the button / key input to conveniently perform the operation of the terminal by using the touch screen. In recent years, not only simple input but also user interface (UI) Is becoming an important function of Accordingly, as the touch function is applied to a mobile terminal in various forms, development of a user interface (UI) corresponding thereto is further demanded.

Various functions and user interfaces implemented in the mobile terminal provide various menus for convenience of input.

In general, augmented reality is a technique for superimposing virtual objects on the real world viewed by the user. In other words, augmented reality is basically based on real world and putting virtual information on it. In this way, augmented reality technology, which is a mixture of real environment and virtual objects, has the advantage of providing additional information in addition to the actual environment.

Researches on augmented reality technology have been carried out since the start of research on augmented reality technology. Recently, studies have been actively carried out to combine the augmented reality with development of a smartphone. Recently, the improvement of the computing capacity of mobile devices such as mobile terminals, personal digital assistants (PDAs) and ultra mobile personal computers (UMPS), and the development of wireless network devices have made possible handheld enhancement type systems.

For example, if you illuminate the surrounding area with a smartphone camera, information about the building is displayed on the screen when the information of the nearby shops (shop name, location, phone number and other menu) is displayed in stereoscopic image.

However, despite the fact that the spread of mobile devices such as smart phones has rapidly become widespread and an environment for accessing the enhancement room has been established, a variety of applications and user interfaces (UI) using the augmented reality have not been developed.

An object of the present invention is to provide a virtual memo management method of a mobile terminal capable of displaying and registering a virtual memo at a specific space position and displaying the registered memo when the camera is lit.

According to an embodiment of the present invention, there is provided a virtual memo management method for a mobile terminal, the virtual memo management method comprising: creating a virtual memo at a specific space position on a preview screen of a camera when executing a virtual memo application; Registering the created virtual memo in a server; And displaying the registered virtual memo at a corresponding spatial location if the camera is directed to a specific spatial location during the replay of the virtual memo application.

The spatial position is an azimuth angle that makes a specific object or a specific object included in the preview screen.

The virtual memo is created on the memo board, and the memo board is displayed when the main menu of the virtual memo is selected or an object to which the virtual memo is to be attached is touched.

The virtual memo includes a memo content, a location of the user, and an azimuth angle toward the spatial location.

If there are a plurality of virtual memos to be displayed, they are displayed for each creator by date, and are displayed to be distinguished from virtual memos of other users.

In the created virtual memo, security information indicating whether the memo content is disclosed or not is set.

In order to display the virtual memo at a spatial position, the virtual memo is transmitted to the server by checking the position and the azimuth of the user, receiving virtual memo and guide information corresponding to the user's position and azimuth angle from the server, And displays the virtual memo previously registered in the space position.

The guide information includes information indicating that the preview screen of the camera is directed to a space position where the virtual memo is attached, and includes a display, a photograph, and a best shot information for directing the direction of the camera.

The location of the user is measured by using a PGS signal in outdoor, a location tracking using at least three WiFi APs in a room, a location tracking using a unique ID of a home appliance using a local communication, The tracking of the trajectory and the RSSI signal strength measurement of the base station.

According to another aspect of the present invention, there is provided a mobile terminal including a display unit displaying a preview screen of a camera when a virtual memo application is executed, And a control unit for registering the memo in the server and displaying the registered virtual memo at the corresponding position when the camera faces the specific space position at the time of re-executing the virtual memo application.

The spatial position is an azimuth angle that makes a specific object or a specific object included in the preview screen.

The virtual memo is created on the memo board, and the memo board is displayed when the main menu of the virtual memo is selected or an object to which the virtual memo is to be attached is touched.

The virtual memo includes a memo content, a location of the user, and an azimuth angle toward the spatial location.

If there are a plurality of virtual memos to be displayed, they are displayed for each creator by date, and are displayed to be distinguished from virtual memos of other users.

In the created virtual memo, security information indicating whether the memo content is disclosed or not is set.

In order to display the virtual memo at a spatial position, the control unit confirms the position and the azimuth of the user and transmits the virtual memo to the server, receives virtual memo and guide information corresponding to the user's position and azimuth from the server, On the preview screen, and displays the virtual memo previously registered in the space position.

The guide information includes information indicating that the preview screen of the camera is directed to a space position where the virtual memo is attached, and includes a display, a photograph, and a best shot information for directing the direction of the camera.

The control unit confirms the location of the user through the location tracking using the PGS signal in the outdoor, the location tracking using at least three WiFi APs in the indoor, the location tracking using the unique ID of the home appliance using the short distance communication, And the RSSI signal strength measurement of the base station.

The mobile terminal having the virtual memo function and the virtual memo management method according to the present embodiment create and record a memo at a specific space position and then display the created virtual memo when the user faces the corresponding space position of the mobile terminal So that memos can be easily created and checked regardless of place and time. In particular, when a virtual memo application is widely used and a user's virtual memo is sufficiently accumulated, it can be searched in the form of a database through the Internet, and the user can also use the virtual memo application.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention;
2 is a block diagram of a wireless communication system in which a mobile terminal may operate in accordance with an embodiment of the present invention.
3 is a view showing an example of creating a virtual memo after selecting an object in the embodiment of the present invention;
4 is a view showing an example of selecting an object after creating a virtual memo in the embodiment of the present invention;
FIG. 5 illustrates a form in which a plurality of memos are added to one object in the embodiment of the present invention. FIG.
6A and 6B are views showing an example of guide information displayed on a preview screen of a camera in an embodiment of the present invention.
7 illustrates an example of setting security for a virtual memo in an embodiment of the present invention;
8 is a diagram showing an example in which a user B confirms a virtual memo created by the user A in the embodiment of the present invention.
9 is a diagram illustrating an example in which a user A directly transmits virtual memo information to a user B to inform the user of the position of the virtual memo in the embodiment of the present invention.
10 is a flowchart illustrating a method for managing a virtual memo of a mobile terminal according to an embodiment of the present invention.

Hereinafter, a mobile terminal related to the present invention will be described in detail with reference to the drawings. The suffix "module" and " part "for components used in the following description are given merely for convenience of description, and do not give special significance or role in themselves. Accordingly, it should be noted that the above-mentioned "module" and "part"

The terminal may be implemented in various forms. For example, the terminal described in this specification may include a mobile terminal such as a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player) And fixed terminals such as a digital TV, a desktop computer, and the like. In the following description, it is assumed that the terminal is a mobile terminal. However, it will be readily apparent to those skilled in the art that the configuration according to the following description may be applied to the fixed terminal, except for components specifically configured for mobile use.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an audio / video input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, A controller 170, a controller 180, a power supply 190, and the like. 1 shows a mobile terminal having various components. However, not all illustrated components are required. A mobile terminal may be implemented by more components than the illustrated components, or a mobile terminal may be implemented by fewer components.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more components that allow wireless communication between the mobile terminal 100 and the wireless communication system or wireless communication between the mobile terminal 100 and a network on which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

Meanwhile, the broadcast related information may be provided through a mobile communication network, and in this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcasting signals using various broadcasting systems. In particular, the broadcasting receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S) Only Digital Broadcast-Handheld (DVB-H), Integrated Services Digital Broadcast-Terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 is configured to be suitable for all broadcasting systems that provide broadcasting signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

In addition, the mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data depending on a voice call signal, a video call signal, or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and the wireless Internet module 113 can be built in or externally. WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, and the like can be used as the short distance communication technology.

The location information module 115 is a module for confirming or obtaining the location of the mobile terminal. A typical example of the position information module 115 is a Global Position System (GPS) module. According to the present technology, the GPS module calculates distance information and accurate time information from three or more satellites, and then applies trigonometry to the calculated information to obtain three-dimensional current position information according to latitude, longitude, and altitude It can be calculated accurately. At present, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another satellite is widely used. In addition, the GPS module can calculate speed information by continuously calculating the current position in real time.

An audio / video (A / V) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes an image frame such as a still image or a moving image obtained by the image sensor in the video communication mode or the photographing mode. Then, the processed image frame can be displayed on the display module 151.

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ The camera 121 may be equipped with two or more cameras according to the configuration of the terminal.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. The microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a key pad, a dome switch, a touch pad (static / static), a jog wheel, a jog switch, and the like. Particularly, when the touch pad has a mutual layer structure with the display module 151 described later, it can be called a touch screen.

The sensing unit 140 senses the current state of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence or absence of user contact, the orientation of the mobile terminal, And generates a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. Also, it is responsible for a sensing function related to whether or not the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor 141. This will be discussed later in connection with touch screens.

The sensing unit 140 includes a geomagnetic sensor for calculating the direction of movement when the user moves, a gyro sensor for calculating the direction of rotation, and an acceleration sensor for calculating the moving distance by multiplying the stride of the user by one step .

The interface unit 170 serves as an interface with all external devices connected to the mobile terminal 100. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, Video input / output (I / O) ports, earphone ports, and the like.

Here, the identification module is a chip that stores various information for authenticating the use right of the mobile terminal 100, and includes a user identification module (UIM), a subscriber identity module (SIM) ), A Universal Subscriber Identity Module (" USIM "), and the like. In addition, an apparatus having an identification module (hereinafter referred to as 'identification device') can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port. The interface unit 170 receives data from an external device or receives power from the external device to transfer the data to each component in the mobile terminal 100 or to transmit data in the mobile terminal 100 to an external device.

The interface unit 170 may be a path through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, Various command signals may be transmitted to the mobile terminal. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal is correctly mounted on the cradle.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 150 may include a display module 151, an audio output module 152, and an alarm unit 153.

The display module 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. If the mobile terminal 100 is in the video communication mode or the photographing mode, the captured image and / or the received image or UI and GUI are displayed.

Meanwhile, as described above, when the display module 151 and the touch pad have a mutual layer structure to constitute a touch screen, the display module 151 can be used as an input device in addition to the output device. The display module 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display 3D display). Some of these displays can be configured to be transparent so that they can be viewed from outside. This may be referred to as a transparent display. A typical example of the transparent display is a transparent organic light emitting diode (TOLED) or the like. In addition, there may be two or more display modules 151 according to the embodiment of the mobile terminal 100. For example, the mobile terminal 100 may be provided with an external display module (not shown) and an internal display module (not shown) at the same time. The touch screen may be configured to detect not only the touch input position and area but also the touch input pressure.

The audio output module 152 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 outputs an acoustic signal related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The sound output module 152 may include a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for informing occurrence of an event in a form other than an audio signal or a video signal. For example, it is possible to output a signal in a vibration mode. When a call signal is received or a message is received, the alarm unit 153 can output a vibration to notify it. Alternatively, when the key signal is input, the alarm unit 153 can output the vibration by the feedback to the key signal input. The user can recognize the occurrence of an event through the vibration output as described above. Of course, a signal for notifying the occurrence of an event may also be output through the display module 151 or the sound output module 152. [

The memory 160 may store a program for processing and controlling the control unit 180 and may store a function for temporarily storing input / output data (e.g., a phone book, a message, a still image, . In addition, the memory 160 may store data on vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM (Random Access Memory) SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read- And an optical disc. Also, the mobile terminal 100 may operate a web storage for performing a storage function of the memory 160 on the Internet.

The control unit 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. In addition, the control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives external power and internal power under the control of the control unit 180 as a battery, and supplies power necessary for operation of the respective components.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. In some cases, And may be implemented by the control unit 180.

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. In addition, the software codes may be stored in the memory 160 and executed by the control unit 180. [

In the foregoing, the mobile terminal related to the present invention has been described in terms of components according to functions. Hereinafter, the mobile terminal related to the present invention will be further described with reference to FIGS. 2 and 3 in view of the components according to the outline. Hereinafter, for simplicity of explanation, a slider type mobile terminal will be described as an example among various types of mobile terminals such as a folder type, a bar type, a swing type, a slider type, and the like. Therefore, the present invention is not limited to the slider type mobile terminal but can be applied to all types of mobile terminals including the above-mentioned type.

The terminal 100 shown in Fig. 1 is configured to be operable in a communication system capable of transmitting data through a frame or packet, including a wired / wireless communication system and a satellite-based communication system .

Hereinafter, a communication system capable of operating a terminal according to the present invention will be described with reference to FIG.

The communication system may use different air interfaces and / or physical layers. For example, wireless interfaces that can be used by communication systems include, but are not limited to, Frequency Division Multiple Access ('FDMA'), Time Division Multiple Access ('TDMA'), Code Division Multiple Access (CDMA), Universal Mobile Telecommunications Systems (UMTS) (especially Long Term Evolution (LTE)), Global System for Mobile Communications (GSM) . Hereinafter, for the sake of convenience of description, the description will be limited to CDMA. However, the present invention is applicable to all communication systems including CDMA wireless communication systems.

2, the CDMA wireless communication system includes a plurality of terminals 100, a plurality of base stations (BSs) 270, and base station controllers (BSCs) 275 , And a mobile switching center ('MSC') 280. MSC 280 is configured to be coupled to a Public Switched Telephone Network (" PSTN ") 290, and is also configured to be coupled to BSCs 275. BSCs 275 may be coupled in pairs with BS 270 through a backhaul line. The backhaul line may be provided according to at least one of E1 / T1, ATM, IP, PPP, Frame Relay, HDSL, ADSL or xDSL. Thus, a plurality of BSCs 275 may be included in the system shown in FIG.

Each BS 270 may comprise at least one sector, and each sector may comprise an omnidirectional antenna or an antenna pointing to a particular direction of radial from BS 270. [ In addition, each sector may include two or more antennas of various types. Each BS 270 may be configured to support a plurality of frequency assignments and each of the plurality of frequency assignments has a specific spectrum (e.g., 1.25 MHz, 5 MHz, etc.).

The intersection of sector and frequency assignment may be referred to as a CDMA channel. BS 270 may be referred to as a Base Station Transceiver Subsystem (BTSs). In this case, the word "base station" may be referred to as a combination of one BSC 275 and at least one BS 270. [ The base station may also indicate a "cell site ". Alternatively, each of the plurality of sectors for a particular BS 270 may be referred to as a plurality of cell sites.

2, a broadcasting transmitter (BT) 295 transmits a broadcasting signal to the terminals 100 operating in the system. The broadcast receiving module 111 shown in FIG. 1 is provided in the terminal 100 to receive a broadcast signal transmitted by the BT 295.

In addition, FIG. 2 illustrates several Global Positioning System ('GPS') satellites 300. The satellites 300 help to locate at least one of the plurality of terminals 100. Although two satellites are shown in FIG. 2, useful location information may be obtained by two or more satellites. The location information module 115 shown in FIG. 1 cooperates with satellites 300 to obtain desired location information. Here, you can track your location using all of the technologies that can track your location, as well as your GPS tracking technology. Also, at least one of the GPS satellites 300 may optionally or additionally be responsible for satellite DMB transmission.

Among the typical operations of a wireless communication system, the BS 270 receives a reverse link signal from the various terminals 100. At this time, the terminals 100 are connecting a call, transmitting or receiving a message, or performing another communication operation. Each of the reverse link signals received by the particular base station 270 is processed by the particular base station 270. The data resulting from the processing is transmitted to the connected BSC 275. The BSC 275 provides call resource allocation and mobility management functions, including the organization of soft handoffs between the base stations 270. The BSC 275 also transmits the received data to the MSC 280 and the MSC 280 provides additional transmission services for connection with the PSTN 290. [ Similarly, the PSTN 290 connects to the MSC 280, the MSC 280 connects to the BSCs 275, and the BSCs 275 communicate with the BSs 270 ).

Generally, when the smartphone camera is operated in a preview state, the augmented reality displays virtual information about the object, that is, information (position, name, phone number, and the like) of the object and information related to the object on the preview screen of the actual object It is.

The present invention provides an application for displaying a created virtual memo together with augmented reality information when a virtual memo is created and stored in a specific space using a function of the augmented reality and then the position is viewed by a smartphone camera. In particular, the present invention provides a method for creating, registering, and displaying a virtual memo using indoor and outdoor augmented reality information by applying GPS, geomagnetic sensor, and various user's location tracking method. The virtual memo is abbreviated as a memo for convenience of explanation.

1. Registration of user's representative location

The user executes the virtual memo application to create a virtual memo. The user can set his / her representative position in the main menu of the executed virtual application. The representative position of the user is used to detect the azimuth angle of the fixture when the specific fixture is illuminated using the mobile terminal (or smart phone), and to display the memo about the fixture at a more accurate position. That is, when the location tracking through the GPS and the WiFi AP is not good, the memo can be located at a more accurate position based on the registered representative position.

The representative location of the user is detected by using the GPS signal in the location information module 115 outdoors and by using a wireless LAN (WiFi) access point (AP) in the room.

The detected representative position of the user is registered in the virtual memo management server. The virtual memo management server manages not only the representative location but also memos and related information of users subscribed to the virtual memo application. At this time, the location registered by the user is a location where the user spends a lot of time. For example, the user includes a fixture (a living room, a wall, a window, a refrigerator and an air conditioner) May include a library or a specific meeting place that is frequently visited.

In particular, the representative position of the user is used as a reference position only when the position measurement of the user (mobile terminal) is good in indoor and outdoor.

2. Create a virtual note

When the virtual memo application is executed, the camera, the position information module, and various sensors (geomagnetic sensor, gyro sensor, etc.) are automatically operated.

The camera captures an object to leave a memo. The geomagnetism sensor measures an azimuth angle of the mobile terminal toward an object to which a memo is to be left. The gyro sensor performs a role of specifying a movement locus of the mobile user.

In a state in which the camera is operated and at least one object is displayed on the preview screen, the user can create and register a note on the object or an azimuth angle toward the object. In other words, the user must decide what type to write notes to.

Whether to leave the memo in a predetermined object (fixture) or in the azimuth can be provided as a separate menu when the main menu or the screen of the virtual memo application is long-touched.

3 shows an example of creating a virtual memo after selecting an object in the embodiment of the present invention.

As shown in FIG. 3, when the user touches the preview screen for a long time, a menu for selecting an object or an azimuth angle in a pop-up form is provided. When the user selects the object or the azimuth angle from the menu, the memo board 50 is displayed on the preview screen.

The user creates a virtual memo on the memo board displayed above.

A method for creating a virtual memo on the memo board 50 is divided into two types.

In the first method, a virtual memo is created in the memo board and registered immediately. In the second method, a memo board is created to create a virtual memo. Then, the created memo is moved to a desired object by a touch & How to register. That is, the first method is a method of creating a virtual memo after selecting an object, and the second method is a method of selecting an object to be left after a memo is written first.

3, the user may, for example, touch the refrigerator for a long time to display a memo board, create a virtual memo on the displayed memo board, and then click 'OK' on the user input unit 130 of the mobile terminal. Button to indicate that the creation of the virtual memo is completed. When the confirmation button is input, the controller 180 adds a note to the selected refrigerator.

In the second method, as shown in FIG. 4, a virtual memo is created in the (virtual) memo board, a menu for generating a new memo is displayed by touching the lower part of the main board, To add the virtual memo to the desired object. At this time, the menu includes new memo adding, deleting and moving yielding.

When the virtual memo is created, the user selects the new memo addition item. When the corresponding item is selected, the created virtual memo becomes a small memo. The user adds the virtual memo to the refrigerator by touching and dragging the changed small memo, for example, by placing the memo in the refrigerator.

5 is a view showing an example in which a plurality of memos are added to one object in the embodiment of the present invention.

As shown in FIG. 5, the virtual memo can be created and added to one object or azimuth angle. In addition, it is possible to select an additional item of the virtual memo application or select a memo adding icon (clip form) provided in the memo board so as to further include at least one virtual memo in one virtual memo. 3. Virtual memo registration

The registration of the virtual memo takes place at the time when the virtual memo is added to a specific object or azimuth. When the virtual memo is added to a specific object or azimuth, the controller 180 stores the position of the user measured in the position information module 115, the azimuth measured at the geomagnetism center, and the contents of the virtual memo in the virtual memo management server ).

The virtual memo management server is a server for managing virtual memos of users subscribed to the virtual memo application, and can be interlocked with an existing location information server or augmented reality information server. Therefore, the virtual memo may be displayed in association with existing augmented reality information, or may display only a virtual memo to a user subscribed to the virtual memo application by the virtual memo management server.

The created virtual memo is transmitted to the virtual memo management server and stored in the virtual memo box of the mobile terminal. The stored virtual memo also includes a location of the mobile terminal, an azimuth angle of the mobile terminal toward the object, and a memo content.

The virtual memo stored in the virtual memo box is stored in the form of a folder or an icon, and includes an azimuth angle, a position, and a memo content. In addition, the memo includes a title, a memo date, and time information.

3. Show notes

The display of the virtual memo is displayed when the user places the camera at a specific object or azimuth after executing the virtual memo application.

When the virtual memo application is executed, the controller 180 grasps the current position of the mobile terminal using the GPS signals received from the GPS satellites through the position information module 115, And measures the azimuth angle of the current mobile terminal.

The measured position and azimuth angle of the mobile terminal are transmitted to the virtual memo management server, and the virtual memo management server transmits at least one virtual memo corresponding to the position and the azimuth angle to the mobile terminal. Accordingly, the control unit 180 superimposes the virtual memo transmitted from the virtual memo management server on a preview screen (live image) currently displayed on the mobile terminal, for example, a predetermined object or azimuth.

The virtual memo displayed at this time displays the date (or time) or the creator of the creation of the virtual memo. If there are a plurality of memos, the virtual memo is displayed for each creation date or for each creator.

The most important factor when displaying a virtual memo on the preview screen of the mobile terminal is the current position of the mobile terminal, that is, the position of the user. In the case of outdoor, it is possible to grasp the position of the user relatively accurately by using the GPS signal, but in the case of the indoor, various methods are used to determine the position.

For example, after measuring the current position of the mobile terminal using three or more WiFi APs in or around the home, the gyro sensor provided in the mobile terminal measures the position of the mobile terminal based on the movement trajectory of the user And corrects the measured position using the received signal strength (RSSI) of the base station.

As another example, the position of the mobile terminal can be determined by comparing the image photographed by the mobile terminal with the design of the home registered in the virtual memo management server using the image recognition, and the unique ID of the home appliances using the short- It is possible to track and correct the position of the mobile terminal.

When displaying the registered virtual memo, the virtual memo management server can provide the guide information so that the user can more accurately direct the object to which the current memo is attached. The virtual memo management server provides the guide information as shown in FIGS. 6A and 6B so that the mobile terminal faces the object or azimuth angle of the memo based on the tracked position of the mobile terminal and the angle of the mobile terminal.

In addition, the virtual memo management server may provide the mobile terminal with a photograph or best shot location information of the object in which the memo is left, in addition to the guide information. The control unit 180 can notify the user that the optimal position has been reached when the preview screen of the user coincides with the photograph or best shot position information.

Accordingly, the user adjusts the direction of the mobile terminal based on the guide information displayed on the preview screen of the mobile terminal, and places the object (e.g., refrigerator) fmf remaining in the virtual memo on the preview screen with accuracy.

In this manner, the user can easily confirm the virtual memo added to the azimuth angle of the specific object by himself / herself using his / her mobile terminal.

In the present invention, the virtual memo can be shared with other users using the virtual memo application. In general, a memo is related to the user's own schedule, but it is also used to notify other users of specific facts, so sharing with other users is necessary.

After creating a virtual memo, the user can set security for the virtual memo in the setting menu. Security settings are classified as public or private, and public users can view virtual memos created by others.

7 is an example of setting security for a virtual memo in the embodiment of the present invention.

The user can add the user list in the security setting menu, and can set the user who wants to disclose the virtual memo in the user list. Accordingly, the user whose virtual memo disclosure is checked can view the virtual memo added by another user to a specific object or azimuth by selecting the virtual memo view after executing the virtual memo application.

8 is a diagram showing an example in which the user B confirms the virtual memo created by the user A in the embodiment of the present invention. At this time, the user A and the user B live in the same house, and the user A creates a virtual memo as an example.

First, user A attaches the virtual memo to the refrigerator and sets security so that user B can see it. The created virtual memo is registered in the virtual memo management server together with the location information and the azimuth of the mobile terminal.

When the user B then returns to the virtual memo application and selects the memo view, the control unit 180 of the user B transmits a signal for requesting the virtual memo to the virtual memo management server. The virtual memo request signal includes the location information of the user B. Accordingly, the virtual memo management server checks whether the user B is a user who is set to release the memo, and transmits the virtual memo registered by the user to the user B's mobile terminal if the memo disclosure is set.

Accordingly, the user B recognizes the virtual memo of the user A by allowing the mobile terminal to operate in the refrigerator according to the guide information provided by the virtual memo management server, and selects the virtual memo to check the contents of the detailed memo.

6A and 6B, the control unit 180 displays a guide information for guiding the position (object or azimuth angle) of the memo, as shown in FIGS. 6A and 6B, according to the virtual memo information transmitted from the virtual memo management server. . Accordingly, the user B can accurately recognize where the virtual memo left by the user A is, and can confirm the virtual memo attached to the object or the azimuth angle.

On the other hand, when the user B shines the mobile terminal on the refrigerator, a plurality of memos may exist. If a plurality of virtual memos are attached to the refrigerator and the virtual memo is not pasted by the user B, the control unit 180 gives a special effect (flicker, different color, movement and sound) so as to be distinguished from the virtual memo of the user B . In addition, the urgency of the virtual memo or the virtual memo past the time changes the color and size differently from other virtual memos.

9 is a diagram illustrating an example in which a user A directly transmits virtual memo information to a user B to inform the user of the location of the virtual memo in the embodiment of the present invention.

Referring to FIG. 9, the user A transmits a virtual memo to the user B so that the user B can confirm the virtual memo at the location where the user A has set the virtual memo. For example, if it is desired to inform the user B of the third floor of a certain building (eg, Starbucks) at a certain point (A) in the outdoors, the user A makes a virtual memo at the azimuth toward the third floor of Starbucks from the above point (A) And transmits the registered virtual memo information to the user B. The transmitted virtual memo information includes the memo content, the azimuth angle, and the location of the user.

Accordingly, the user B receives the GPS guidance and reaches the Starbucks. Based on the virtual memo information (the position and the azimuth of the user A) received from the user A, Confirm the virtual memo left by user A towards the layer.

10 is a flowchart illustrating a method for managing a virtual memo of a mobile terminal according to an embodiment of the present invention.

As shown in FIG. 10, in order to create a virtual memo, the user first executes a virtual memo application (S10). When the virtual memo application is executed, the camera, the position information module 115 and the geomagnetism sensor are automatically operated (S11).

When the user selects the setting menu of the main menu of the virtual memo or touches the object long by touching the preview screen of the camera with the object (object, fixed object) to which the virtual memo is to be left, If the user decides to leave a virtual memo on a specific object in the pop-up, the control unit 1800 inputs the memo content to the preview screen A memo board can be displayed.

In the displayed memo board, the user creates a memo content, and if necessary, performs another memo, or includes a plurality of memos in a memo (S12). In addition, the user sets security for the created virtual memo and sets whether or not the created virtual memo is disclosed.

When the user attaches the created virtual memo to a specific object, the controller 180 transmits the memo information including the memo contents, the user's location and the azimuth angle to the virtual memo management server and registers the memo in step S313.

When the user then executes the virtual memo application again and selects the memo view, the controller 180 measures the current position of the user using the various position measuring methods described above, and then measures the position of the user measured by the geomagnetism sensor The azimuth angle is included in the virtual memo request signal and transmitted to the virtual memo management server (S14, S15).

 The virtual memo management server searches at least one virtual memo from the database based on the user's location and azimuth angle transmitted from the mobile terminal, and transmits the retrieved virtual memo together with the guide information to the mobile terminal.

Accordingly, the control unit 180 of the mobile terminal displays the guide information received from the virtual memo management server on the preview screen to guide the object attached with the memo, and display the virtual memo created on the object. At this time, if there is a virtual memo pasted by another user in the object, the virtual memo is marked to be distinguished.

In addition, in the present invention, not only a text but also a photograph, a movie, an application shortcut, a telephone number, and the like can be set in the virtual memo. In association with the schedule and the alarm function, a specific memo is activated at a time designated by the user, Or the like.

As described above, according to the present invention, when a user creates a memo at a specific space and registers the virtual memo, the user can easily write a memo Can be confirmed. Especially, when the virtual memo application is used in various ways and the user's virtual memo is sufficiently accumulated, it can be retrieved by searching the database through the Internet, and it can be used for advertisement.

That is, users can check which memo is located at a specific location using the wireless Internet, and can check whether a person left a virtual memo at the location. These virtual memos can be used in various ways in the future as more and more memories are made by leaving the lovers' love oaths and love letters in specific areas (Namsan, famous places and restaurants) and memorizing them.

Further, according to an embodiment of the present invention, the above-described method can be implemented as computer-readable code on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer-readable medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and also implemented in the form of a carrier wave (for example, transmission over the Internet) . Further, the computer may include a control unit of the terminal.

The mobile terminal described above can be applied to not only the configuration and method of the embodiments described above but also all or some of the embodiments may be selectively combined so that various modifications may be made to the embodiments It is possible.

110: wireless communication unit 111: broadcast receiving module
112: mobile communication module 113; Wireless Internet Module
140; Sensing unit 150: output unit
151: display unit 160: memory
180:

Claims (24)

Creating a virtual memo at an azimuth toward a specific space position on a preview screen of the camera when the virtual memo application is executed;
Registering the created virtual memo, azimuth and user location on the server;
Receiving virtual memo information from a server when replaying the virtual memo application and displaying guide information for guiding the attachment position of the virtual memo on the preview screen; And
And displaying the virtual memo registered at the azimuth angle when the preview screen is moved based on the displayed guide information and the registered azimuth angle is reached. The method of claim 1,
At least one azimuth angle toward a specific space position or at a specific space position, and each virtual memo can include a plurality of different virtual memos. 2. The method of claim 1,
Wherein the object is an object included in a preview screen. The method of claim 1,
Wherein the memo board is displayed when the main menu of the virtual memo is selected or when an object to which the virtual memo is to be attached is touched. The method of claim 1,
The azimuth angle or the object is selected or attached when the virtual memo is created and then moved to a specific azimuth or object by drag and drop. delete The method according to claim 1, wherein when there are a plurality of virtual memos
Wherein the virtual memo management information is displayed for each author by date. The method according to claim 1, wherein when there are a plurality of virtual memos
And displaying the virtual memo to be distinguished from the virtual memo of the other user. The method of claim 1, further comprising: setting security information indicating whether or not the memo content is disclosed in the created virtual memo. The method of claim 1, further comprising: transmitting virtual memo information including the created virtual memo, the azimuth angle, and the user location to another user; And
Displaying guide information for guiding an azimuth angle of the virtual memo at the current user's position and angle based on the transmitted virtual memo information when another user executes the virtual memo application; And
And displaying the virtual memo set at the azimuth angle when the preview screen moves to the predetermined azimuth angle according to the displayed guide information. 11. The method of claim 10, wherein the guide information
Wherein the virtual memo management information includes information indicating that a preview screen of the camera is directed to a space position where the virtual memo is attached, and includes a display, a photograph, and a best shot information for directing the direction of the camera. 11. The method of claim 10, wherein the location of the user is
In the outdoor, it is measured by using GPS signal, location tracking using at least three WiFi APs in indoor, and location tracking using home ID of home appliances using short distance communication. The RSSI signal strength of the mobile terminal is corrected through the RSSI signal strength measurement of the mobile terminal. A display unit for displaying a preview screen of the camera when the virtual memo application is executed; And
When a virtual memo is created in the azimuth direction toward the specific space position of the preview screen, the virtual memo is registered in the server, the virtual memo information is received from the server in the replay of the virtual memo application, And displaying the virtual memo registered at the azimuth angle when the preview screen is moved based on the displayed guide information and the registered azimuth angle is moved. 14. The method of claim 13,
At least one azimuth angle toward a specific space position or at a specific space position, and each virtual memo can include a plurality of different virtual memos. 14. The method of claim 13,
And an object included in the preview screen. 14. The method of claim 13,
Wherein the memo board is displayed when a main menu of a virtual memo is selected or when an object to which a virtual memo is to be attached is touched. 14. The method of claim 13,
The azimuth angle or the object is selected or attached when the virtual memo is created and then moved to a specific azimuth or object by drag and drop. delete 14. The apparatus of claim 13, wherein the control unit
And when the virtual memo is a plurality of virtual memos, the virtual memo is displayed for each creator by date. 14. The apparatus of claim 13, wherein the control unit
And displays the virtual memo so as to be distinguishable from the virtual memo of another user when the virtual memo has a plurality of virtual memos. 14. The mobile terminal of claim 13, wherein security information indicating whether the memo content is disclosed is set in the created virtual memo. 14. The apparatus of claim 13, wherein the control unit
The virtual memo information including the created virtual memo, the azimuth angle, and the user location is transmitted to another user,
The terminal of another user displays guide information for guiding the azimuth with the virtual memo at the position and angle of the current user on the basis of the virtual memo information when the virtual memo application is executed and the preview screen is moved according to the displayed guide information, And displays the virtual memo set at the azimuth angle when the mobile terminal is directed to the azimuth angle. 23. The method of claim 22,
And information indicating that the preview screen of the camera is directed to the spatial position to which the virtual memo is attached, and includes a display, a photograph, and a best shot information for instructing the direction of the camera to move. 23. The apparatus of claim 22, wherein the control unit
In the outdoor, it is possible to confirm the location of the user through location tracking using the GPS signal, location tracking using at least three WiFi APs in the room, location tracking using the unique ID of home appliances using the short distance communication, Tracking the trajectory and measuring the RSSI signal strength of the base station.

Images