KR20160090938A - Method for searching information using text in taken picture - Google Patents

Abstract

The present invention relates to a search method using text in a photographed photo. According to an embodiment of the present invention, disclosed is a search method using text in a photographed photo, the search method comprising the steps of: invoking, by a user, a search window; operating a camera by referring to a region, set by the user, in response to the invocation; increasing visibility by adjusting transparency of a region exclusive of the search window, and inputting, by the user, a search word to the search window; and reading text inside an image visible in a camera region, selecting a search word among a plurality of generated search words based on the text, and performing search based on the selected search word. Additionally, another configuration can be provided according to technical spirit of the present invention.

Description

METHOD FOR SEARCHING INFORMATION USING TEXT IN TAKEN PICTURE [0002]

In particular, the present invention relates to a method and an interface for inputting or selecting various keywords, texts, and images in the vicinity of an input box using a camera in combination with an existing search, .

Recently, various communication and sensing functions such as smart phone and smart pad have been developed.

The use of the Internet using mobile smart devices with powerful computing functions is increasing. However, the information retrieval method for obtaining the necessary information by the user is limited to the method of using the search engine based on the existing text. Although some image-based searches are known, they have not been widely used due to inconvenience and low accuracy.

In such a situation, it is necessary to search for characters, texts and numbers in daily life, to reduce the hassle of inputting them into the input window separately, and to search for faster and easier. However, according to the prior art, the (potential) intention of such a user could not be adequately supported.

SUMMARY OF THE INVENTION The present invention has been made to solve all the problems of the prior art described above.

An object of the present invention is to input or select various keywords, texts, and images in combination with an existing search.

In order to accomplish the above object, a representative structure of the present invention is as follows.

According to an aspect of the present invention, there is provided a search method using text in photographed photographs, the search method comprising: calling a search window by a user; In response to the call, operating the camera with reference to the area set by the user; The area other than the search window is increased in transparency to improve the discrimination. Inputting a search word by a user into the search window; The text in the image displayed in the camera area is read and a search word is selected from a plurality of the search words generated based on the text. Performing a search based on the selected search term; The method of searching using text in a photographed photo is disclosed.

Other configurations may be further provided in accordance with the technical idea of the present invention.

According to the present invention, it is possible to provide an intuitive and convenient input method in various smart devices.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram showing a schematic configuration of an entire system for searching using photographed text according to an embodiment of the present invention; FIG.
2 is a view showing shooting a business card using a smartphone according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating the import of text in a business card using a camera of a smartphone according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating loading a portrait in a photographed image using image recognition in accordance with an embodiment of the present invention. FIG.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

Configuration of the entire system

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram showing a schematic configuration of an entire system for searching using photographed text according to an embodiment of the present invention; FIG.

1, an overall system according to an embodiment of the present invention may be configured to include a communication network 100, a character and image interpretation system 200, and a smart device 310. As shown in FIG.

First, the communication network 100 according to an embodiment of the present invention can be configured without regard to communication modes such as wired communication and wireless communication. The communication network 100 can be a local area network (LAN), a metropolitan area network ), A wide area network (WAN), and the like. Preferably, the communication network 100 referred to herein may be the well-known Internet or World Wide Web (WWW). However, the communication network 100 may include, at least in part, a known wire / wireless data communication network, a known telephone network, or a known wire / wireless television communication network, without being limited thereto.

Next, the character and image analysis system 200 according to an exemplary embodiment of the present invention may be a digital device having memory means and equipped with a microprocessor and having computational capability. The character and image interpretation system 200 may be a server system. The character and image interpretation system 200 may be configured to provide the predetermined information or control command to one of the devices 300 through the communication network 100 or to provide the predetermined information or control command Can be performed.

For this purpose, the character and image analyzing system 200 analyzes the character or image input through the mobile smart device 310, as described in detail below, to the communication network 100 or the smart device 310, Lt; / RTI >

For this purpose, optical character recognition (OCR), which is a method of acquiring images of human-written or machine-printed characters with image scanners and converting them into machine-readable characters, or analyzing image files to find identical or similar images Can be used.

Such a user interface may be provided by a control unit (not shown) included in the character and image analysis system 200. Such control may be in the form of a program module within the content 200. Such program modules may take the form of an operating system, application program modules or other program modules. The program modules may also be stored in a remote storage device that is capable of communicating with the text and image interpretation system 200.

Meanwhile, the program module may perform a specific task, which will be described later,

But are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that execute abstract data types.

In addition, the character and image interpretation system 200 further stores the information about the attitude or movement provided from at least one of the smart devices 300, and further makes it available for use by at least one of the smart devices 300 can do. The storage described above may be performed by a repository (not shown) included by the character and image interpretation system 200. Such a repository is a concept including a computer-readable recording medium, and may be a broad database including a data base based on a file system as well as a consultation database.

Details of the character and image interpretation system 200 will be described in more detail below. Although the above description has been made with respect to the character and image interpretation system 200, this description is exemplary and at least some of the functions or components required by the character and image interpretation system 200 may be subject to manipulation It will be apparent to those skilled in the art that the present invention may be realized or embodied in at least one of the smart devices 310. [

Lastly, the smart device 310 according to an embodiment of the present invention is a digital device including a function of communicating after the connection, and a digital device equipped with a memory means and equipped with a microprocessor for computation May be employed as the smart device 310 according to the present invention. The smart device 310 may be a so-called smart device such as a smart phone, a smart pad, a smart glass, a smart watch, a smart band, smart ring or the like or a desktop computer, a notebook computer, a workstation, a PDA, a web pad, A keyboard, an electronic pen, and the like.

In particular, according to an embodiment of the present invention, the smart device 310 may include at least one technical means capable of receiving an operation from the user. Examples of such technical means include a touch panel, a pointing tool (e.g., a mouse, a stylus, an electronic pen, etc.), a graphical object that can be operated by a user, a keyboard, a toggle switch, A recognition sensor, a distance sensor, and the like.

In addition, according to an embodiment of the present invention, the smart device 310 may include at least one technical means for acquiring physical information about the posture or movement of the smart device 310. [ Gyro, magnetic sensor, positioning module (GPS module, beacon-based positioning module, etc.), a barometer, a distance sensor, Sensors and the like.

Also, the smart device 310 may process the above physical information to provide information or control commands to other smart devices 310, receive information or control commands from other smart devices 310, May further include an application program for generating the application program. Such an application may reside in the form of a program module within the smart device 310. The nature of such a program module may be generally similar to the control of the character and image interpretation system 200 as described above. Here, the application may be replaced with a hardware device or a firmware device, at least some of which can perform substantially the same or equivalent functions as necessary.

An example for this is as follows.

1. Input box with camera input.

-Swipe, input box using Gesture as Trigger, execute the operation to turn on the camera in the limited area. (Camera area can be set by user)

- The area other than the input box can be displayed by adjusting the transparency to check the position of the camera.

- The size of the input box can be wider than the existing text input.

2. Enter the keyword shown in the input box.

- You can select the text in the image displayed in the camera area limited by the input box using multi touch, touch, swipe, etc. Editable and selectable.

- The texts displayed by the camera can be buttoned in semantic units.

- The text displayed in the camera area defined by the input box is displayed as an additional word on the keyboard and can be touched.

- You can use the touch to navigate the area visible by the camera. (You do not have to move the device itself)

3. Create input keyword combinations.

-Trigger If there is text entered by typing at the time of execution, input by combination with keyword which is seen by camera.

-Trigger Execute input in combination with keywords recognized by camera when there is text input by typing after execution.

- The user can edit and organize the typed text to be placed before or after the text recognized by the camera. (Image synthesis and processing)

4. Nearby keyword search and auto-completion feature.

- After inputting keyword by typing, after executing Trigger or Trigger, when typing keyword by typing,

- It can auto-complete and recommend by cropping the area containing the input keyword in the entire area that can be recognized by the camera.

5. Various types of cameras can be used for camera configuration. You can search 360 degrees according to the camera.

6. Trigger through search input type and keyword. (Eg, "camera" + friends, etc.)

7. Determine and auto-complete camera input based on the type of input window and user's history.

- Identify the type of input window and extract the content from the image input by the camera.

- The extracted contents are provided to the user in the input window by the auto completion function.

- Indicators that indicate the direction of the content along with autocomplete

- Analyzes the history that the user frequently input in the input window, analyzes the related contents from the image received from the camera, and provides it with auto completion

8. You can use the camera paste user interface in the input window.

- When you copy & paste to the input car with long touch, the camera paste function is also provided in UI

- When the camera paste function is executed, it automatically provides the appropriate contents of the corresponding input window among the images recognized by the camera in the vicinity.

- Enter the most similar content into the input window and wait.

Fig. 2 is a diagram showing shooting of a business card using a smartphone, according to an embodiment of the present invention. Fig.

Referring first to FIG. 2, the smart device 310 includes a camera 311 on its rear surface, and can output an input window 312 to the front side. At this time, the input window 312 can input characters or input result values derived through the character and image analysis system 200 using the camera 311, or can simultaneously perform the above two operations. The business card 400 is an example of one of various objects from which an image or characters can be extracted, and may include at least one of an image and a character.

FIG. 3 is a diagram illustrating the loading of text in a business card using a camera of a smartphone, according to an embodiment of the present invention. FIG.

The front side is displayed on the screen of the smart device 310 through the camera 311 on the rear side and at this time when characters or a portrait in the business card are input into the input window 312, And the image interpretation system 200 are input into the input window 312 again in the form of a keyword. In this case, although the input window 312 is shown as a long rectangle for easy understanding, various shapes and sizes can be set as necessary.

Fig. 4 is a diagram showing the loading of a portrait in a photographed image using image recognition in an embodiment of the present invention; Fig.

At least two business cards 400 are arranged and captured by the camera, and the result of the character or image interpretation system 200 is output. At this time, when the word 'Photo' corresponding to the word 'Photo' is inputted into the input window 312, the character photographs among the result values are selected and listed through the automatic completion function. When the word 'name' is inputted, When the word 'number' is input, the mobile phone number or the home number among the result values are selected and listed through the automatic completion function.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

Therefore, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the following claims, I will say.

100: Network
200: Character and image interpretation system
310: Smart Device

Claims (3)

A method of searching using optical data,
Inputting optical data; Extracting search data from the optical data; Inputting the extracted search data into an input box; Performing a search based on the input search data;
The method of claim 1, further comprising: The method according to claim 1,
And selecting a range of optical data in the extracting step. The method according to claim 1,
Wherein the text data is generated in the form of a button in the input step to perform input determination and editing.

Images