KR101479771B1 - Information processing system - Google Patents

Abstract

The present invention relates to an information processing system. The present invention provides an information processing apparatus comprising: an input device for inputting information written by a user directly; A recognizing device for recognizing characters, figures and symbols from inputted handwriting contents; A control device for controlling the execution of the application according to the recognized result; Information processing apparatus. According to the information processing system of the present invention, a user interprets and writes contents written directly by using an input device on the basis of characters, figures, symbols, contexts, and the like, Management, telephone directory management, map / local information retrieval, image retrieval, web page retrieval, and the like) are automatically executed to enable processing of desired information.

Sketch, Graffiti, Scheduling, Phone Number, Map / Local Search

Description

INFORMATION PROCESSING SYSTEM

The present invention relates to an information processing system.

Devices for processing various information ranging from a personal computer (PC), a personal digital assistant (PDA), a portable computer (HPC), a portable multimedia player (PMP) An input device is indispensably required for the processing of specific information desired by the user in the information processing device. In most cases, a keyboard or mouse, a touch screen, or a touch pad is used as the input device. Each information processing device can be connected to a server through a wired or wireless network, and can retrieve necessary information from a server and receive it. The type of information to be processed is almost unlimited, and it is possible to search for information of interest such as image search, web search, map / local information search as well as individual schedule management and phone book function. In order to process such information, it is necessary to operate the device. In order to select or execute a specific function or search for the above information, it is necessary to continuously operate the device for executing the function until the final stage do.

The present invention provides an information processing system for analyzing and recognizing information such as characters, figures, symbols, etc. written by hand by a user, and automatically executing an application optimized for the information.

An information processing apparatus according to an embodiment of the present invention includes an input device for inputting information written by a user directly; A recognizing device for recognizing characters, figures and symbols from inputted handwriting contents; A control device for controlling the execution of the application according to the recognized result; And a control unit.

According to an embodiment of the present invention, there is provided an information processing method comprising: recognizing characters, figures, and symbols based on a pattern recognition technique on contents written by a user directly in the form of a sketch or a graffiti using an input device; Analyzing a relationship between at least one of the recognized characters, figures, and symbols to extract meaningful contents; Identifying a user intention according to the extracted contents and executing a corresponding application corresponding to the user intention; And a control unit.

According to the information processing system of the present invention, a user interprets and writes contents written directly by using an input device on the basis of characters, figures, symbols, contexts, and the like, and automatically So that desired information can be processed.

According to the information processing system of the present invention, the intention of the user can be grasped based on the information input in the form of sketch or graffiti, and the function of the device conforming to the user's intention can be executed. .

According to the information processing system according to the present invention, information entered in the form of a sketch or a graffiti is interpreted to automatically execute a schedule management application, automatically execute a phone number management application, Information retrieval can be performed, thereby providing a more convenient and useful user interface and information processing environment.

1 is a view for explaining the structure and concept of an information processing system according to an embodiment of the present invention. The information processing system according to the embodiment can be understood as an information processing unit 100, a service provider 200, and an output unit 300.

The information processing unit 100 includes an input device for inputting information handwritten by a user, a recognition device for recognizing input contents, a control device for executing an appropriate application according to the recognition result, An application related to information processing is mounted. Handwriting by the user manually means inputting information such as letters, figures and symbols using a writing tool such as sketch or graffiti. As an input device that enables this, a device that allows a user to directly record characters, figures, symbols, etc. using a stylus pen, such as a tablet PC 110 or a mobile terminal having a touch screen, can be used. In FIG. 1, for example, all four types of input forms are represented. In the case where the schedule related handwriting 130, the telephone number related handwriting 140, the image related sketch 150, and the map related sketch 160 are in each case . The characteristic of the handwritten information input through the input device is extracted and the extracted characteristic is analyzed to recognize what information is input.

A service provider 200 represents a search portal site-server. In this case, an application for searching information according to the recognized information is an Internet-based information search through a wired or wireless network.

The output unit 300 expresses various output forms of the information processing result. If the recognized information is related to the schedule management, the schedule management information 330 is provided by a schedule management application installed in the apparatus or a server performing schedule management service based on the network. If the recognized information is related to the telephone number, the telephone number information 340 is provided by the phone book management application installed in the device. If the recognized information is related to the image search, the image search result information 350 is provided by a search engine installed in the device or a server performing an image search service based on the network. If the recognized information is related to the local information, the map / local information 360 is provided by a server that performs a map / local information search service based on a search engine or a network installed in the apparatus.

2 is a block diagram illustrating the configuration of an information processing system according to an embodiment of the present invention. The input device 10, the recognition device 20, the control device 30, and the application 40 are included.

As described above, the input device 10 is a device for inputting characters, symbols, figures, and the like by handwriting by hand by the user. The recognition device 20 is a device for recognizing what characters, symbols, and graphics are actually input by analyzing characters, symbols, and figures. The control device 30 selects and executes the most appropriate application using the recognized result, and outputs the execution result through a display device (not shown) or the like.

The application 40 can be understood as a component for performing a corresponding program or a specific function stored in the storage device. For example, the schedule management application 41, the phone book application 42, and the information search application 43 can be mentioned. The schedule management application 41 can be understood as a program for managing an individual schedule in a mobile device such as a mobile phone. The schedule management application 41 may also be provided by a server providing a schedule management service. These services are already implemented, so refer to related technologies. The phone book application 42 can be understood as a program for performing management such as registration, modification, and deletion of a telephone number in a mobile device such as a mobile phone. This function is also provided by the server providing the telephone number management service, and since these services are already performed, refer to related technology. The information search application 43 may be an application that searches for documents, images, multimedia contents, and the like stored in a storage device of a device on which a search engine is installed, or accesses a server based on the Internet to search for information Communication, browsing, and the like. The information to be searched can be variously implemented such as a web page, an image, local information, etc. Here, the image search part 431 and the local information search part 432 are shown as an example.

3 is a diagram showing a configuration of an information recognition section in an information processing system according to an embodiment of the present invention. A variety of techniques can be applied to semantically recognize handwritten content by the user. The information input from the user through the input device 10 corresponds to a kind of sketch or graffiti. Therefore, pattern recognition technology is applied to recognize meaningful information such as characters, symbols, and figures from sketches or graffiti. It extracts features such as handwritten characters, graphic symbols, and the like, extracts features extracted from the extracted features, whether it is a character, a number among characters, a figure such as a line, a circle, a triangle, a rectangle, And the symbol '?'.

There are a variety of techniques for recognizing handwritten handwriting by users as above. In the present invention, a character recognition technology, a shape recognition technology, and a symbol recognition technology that have been already implemented or can be implemented in the future will be used without limiting the recognition device or the recognition method itself.

Referring to FIG. 3, the recognition apparatus according to the embodiment includes at least a character recognition section 21, a code and pattern recognition section 22, and a personal information recognition section 23.

The character recognizing unit 21 recognizes the written character, in particular, a context and a semantic recognizing unit 211 for recognizing the context and a practical meaning, and a place recognizing unit 212 for recognizing whether or not the place name is recognized from the recognized character .

Recognition of characters is based on pattern analysis technology. It is difficult to understand the meaning of input information and user's intention because the arrangement of simple characters recognizes the context and its meaning based on the technique of analyzing the sentence. For example, when recognizing an input character called 'service center' or '02 -555-5555 ', the' service center 'recognizes the context by a combination of words other than a word, and recognizes it as '02 -555-5555' Because the numbers (and symbols) are consecutive, this is recognized as a 'phone number' and consequently the intention to register the phone number of the service center in the phone book in terms of meaning.

The sign and pattern recognition unit 22 includes a figure and shape recognition unit 221, a symbol recognition unit 222, an area recognition unit 223, a size recognition unit 224, a boundary recognition unit 225, A directional recognition unit 226, and a pressure recognition unit 227.

The figure and shape recognition unit 221 recognizes shapes and shapes of lines, circles, triangles, and squares in sketches and graffiti. This can be implemented based on known pattern recognition techniques. In the simplest case, 'line' can be recognized based on the thickness and continuity of the sketched content. If an approximate (relatively thin) thick image is continuous with start and end points, it will be recognized as a 'line'. Alternatively, other known techniques can be used to recognize shapes and shapes of lines, circles, triangles, squares, and the like.

The symbol recognition unit 222 recognizes a special symbol, for example, '?', Which is a symbol expressing a query. The pattern of the user sketch is already in the '?' You will be able to recognize the symbol by comparing how similar it is to the pattern.

The area recognition unit 223 recognizes the area of a specific figure or shape based on the pattern recognition technique. The size recognition unit 224 recognizes the size of the figure or shape based on the pattern recognition technique. The area recognition unit 2250 Recognizes boundaries of figures or shapes based on the pattern recognition technique.

The directional recognition unit 226 recognizes the directionality of the 'line' input by the user through the input device 10, which is an important factor in understanding the intention of the user as will be described later. For example, suppose you have sketched a brief map (map) for local information search. In most cases, you will express 'road' as 'line', and in this case, the direction of east- west- , The left side, the lower side, and the upper side, and it can be inferred that the starting point is the user position or the near side to the user.

The pressure recognition unit 227 senses and recognizes the pressure when the user performs the handwriting through the input device 10. [ The recognized pressure information may be used in various ways. For example, when drawing a graphic object, it is common that the starting point and the ending point apply more pressure than other portions, so that it can be useful information for recognizing whether a sketch or graffiti is a graphic object will be.

The personal information recognizing unit 23 according to the embodiment is represented by a portion 231 for recognizing a user's inclination, taste, and habit. This is particularly useful for acquiring, analyzing and using a user's sketch or graffiti tendencies. Personal information can be constructed by learning patterns that are repeated in a particular way when the user is writing. An example of personal information recognition and use will be described in detail later with reference to FIG.

4 is a diagram illustrating an example of a schedule management information processing method in an information processing system according to an embodiment of the present invention. It is assumed that the user directly writes information related to the schedule through the input device 10. [

4, the schedule-related handwriting 130 is shown. The calendar related handwriting includes a date portion 131, an added symbol and character portion 132, and a content portion 133. The recognition device 20 recognizes the content of the input handwriting. Here, the characters of all the characters are recognized by the character recognition unit 21. [ The context and semantic recognition unit 211 associates the date part 131, the symbol and character part 132 and the content part 133 together with the symbol recognition part 222 in relation to each other and the context and meaning thereof, Is able to recognize that it is '16 days, reinforcement of Japanese lesson' and it is meaningful contents related to schedule management from this. The control device 30 executes the schedule management application 41 and provides (outputs to the display device) the schedule management information 330 in which the schedule 331 of "16 days Japanese reinforcement class" is recorded.

In summary, the recognition of numbers and this was interpreted as a date, recognizing Chinese characters and Hangul, recognizing symbols such as arrows and character inserting codes, and the meaning of reinforcement of Japanese lessons and related with the schedule management application. Update, and edit corresponding contents according to the execution of the schedule management application.

5 is a diagram illustrating an example of a telephone number information processing method in an information processing system according to an embodiment of the present invention. It is assumed that the user directly writes information related to the telephone number through the input device 10. [

The handwriting 140 related to the telephone number is shown in Fig. The calendar-related handwriting includes a name part 141 and a telephone number part 142 to be registered. The recognition device 20 recognizes the content of the input handwriting. The characters recognized by all the characters are recognized by the character recognition unit 21. [ The context part and the semantic recognition part 211 can associate the name part 141 and the telephone number part 142 with each other and recognize that the context and meaning thereof, that is, the user intention is 'service center telephone number registration' It can be seen that the contents are meaningful contents related to the management of the telephone directory. The control device 30 executes the phone book application 42 to register the telephone number "service center, 02-555-5555". That is, the telephone number information 340 is provided and the content 341 is registered therein. If the address is written together, the address can be recognized in the same way and registered automatically.

In summary, the recognition that a string of 'service center' is not a simple character list but a string of a noun type is a meaningful character string, and that the number is recognized as a telephone number, and if an address is input, And recognizes from the relationship of each string that the telephone directory registration is based on the name, the telephone number, and the address. The phone book application is executed according to the result of the context and meaning recognition, and the corresponding item is automatically registered.

In summary, the recognition of numbers and this was interpreted as a date, recognizing Chinese characters and Hangul, recognizing symbols such as arrows and character inserting codes, and the meaning of reinforcement of Japanese lessons and related with the schedule management application. Updating, editing, and the like are performed according to the execution of the schedule management application.

6 is a diagram illustrating an example of an information processing method for image search in an information processing system according to an embodiment of the present invention. The user has sketched the image on the input device 10. The sketched image 150 is recognized by the recognizing device 20 so that the feature is extracted and an image having certain characteristics. Image recognition is based on shape / shape recognition, area recognition, size recognition, boundary recognition, directional recognition, and so on. The feature information extracted from the sketched image is a query for image search. Since the input sketch content is an image, the control device 30 determines that the user has an image search intention and executes the information search application 43. [ Here, the image search application 431 is executed. The image retrieval may be performed from images stored in the device itself or may be performed on an external server based on the Internet. Even if the query is an image, the result of the search may include not only a photograph but also a web page, a document, and text information related to the image. Image retrieval techniques are implemented in various ways. For example, the retrieval technique is based on feature information such as color, shape, and texture of an image. In this case, the search result 350 is provided in descending order of similarity with the shape of the sketched image and the texture feature information.

7 is a diagram illustrating an example of a map-based area information processing method in an information processing system according to an embodiment of the present invention. The user has made a map-related sketch 160 through the input device 10. [ In the map-related sketch 160, building / street names 161, 162, and 163, a search target business name 164, and a search target area 165 are respectively displayed. For map-related sketches, inducing them to use pre-determined forms for roads, buildings, queries, etc. will help them to recognize more quickly and accurately whether they are intent to search map / region-related information. For this purpose, a basic symbol 160a is provided.

The sketched contents are recognized by the recognizing device 20. All the characters are recognized by the character recognition unit 21 and the contents of 'Crown Bakery', 'Pagoda Park', 'Insadong Street', and 'Restaurant Korean Cuisine' are recognized according to context and meaning recognition. Here, the place name recognizing unit 212 recognizes the place name "Insadong". ? "Corresponding to the query is recognized by the symbol recognition unit 222 of the code and pattern recognition unit 22. Thus, it can be seen that the intention is to search for 'Korean restaurant' of 'Insadong'.

The control device 30 executes the local information search application 432 to acquire the Insadong area information 360. [ The local information may be stored in the device itself or may be provided from the server through the Internet.

On the other hand, the figure / shape recognition unit 221, the area recognition unit 223, the size recognition unit 224, and the boundary recognition unit 225 of the recognition apparatus 20 recognize the area 165 to be searched .

First, specify the area by holding the 'Crown Bakery' and 'Pagoda Park' represented on the sketched map as the diagonal end points of the rectangle. Then, the shape of the picture corresponding to the middle of the sketch drawn by the user is grasped. Here, it is recognized as 'circle', and the area and size recognized as the 'circle' will be the range of the search area. The radius of the circle is measured (it will be measured as an approximation since it is a sketch), and a region of a relative size corresponding to the measured circle size on the actual map is selected. The information retrieved from the selected area, which is the information that the user desires to search for, i.e., 'Korean restaurant', is provided and the result is provided.

Therefore, the present invention can receive a map, local information, and a search service on a web or local information providing server through a user's intuitive map entry method. In particular, the contents entered by the user can be accurately grasped on the basis of recognition of characters, recognition and interpretation of symbols and graphics, recognition of directionality, recognition of context, and the like, and information about patterns, tendencies, preferences and habits It is possible to construct a more user-oriented information processing system environment.

The required information can be obtained from a storage device of the device or can be provided based on wired or wireless communication from a system such as a web or other server.

8 is a diagram illustrating an example of a method of grasping a user's input intention in an information processing system according to an embodiment of the present invention. Using the input device, the user painted the line from the upper left corner (P1) to the lower right corner (P2). In the sketched figure, line and direction will be understood by figure and shape recognition and direction recognition. When drawing a line from P1 to P2, the recognition device 20 can guess the following user intention.

1. The sketched image is recognized as pointing to the east, to the west, to the north, and to the south. In other words, it can be assumed that the map expresses 'road'.

2. It can be assumed that the position of the user's starting point is the current user's position or the nearest position.

3. It can be assumed that the direction in which the line travels is the direction that the current user wants to proceed.

Taken together, it is possible to grasp the intention that the user intends to proceed along a road that is directed from P1 to P2. This understanding of the intention is realized by collecting, analyzing, and associating each input information in the same manner with the case of sketching various figures and symbols together. In addition, the shape of a figure has a stereotyped feature, so it can be recognized more easily than a contextual identification. A circle has a center and a radius, a 'square' connects all four segments and has four vertices And the like can be recognized based on the characteristics of the object.

9 is a diagram illustrating an example of a method of grasping user's personal information in an information processing system according to an embodiment of the present invention. It is a method to measure differences between different expressions for each user and to optimize them for the user to take into consideration the acquisition, processing, and retrieval of information. It will also be useful in recognizing information, and it will grasp and use personal information based on adaptive interpretation and learning in information recognition.

FIG. 9 shows a case in which two users perform different sketch-based information input in the same situation. The sketch image A 90a input by the user A and the sketch image B 90b inputted by the user B are shown. The road is represented by a line, and a specific area (range) is marked in the middle of the road. Considering these sketch images 90a and 90b in correspondence with the distances of the actual terrain, it shows the difference in how the actual distances assumed by the user A and the user B are represented on a plane of the same size. For example, considering the extent to which 1 km of radius is actually expressed on the sketch image, the circle area size in the sketch image 90a drawn by the user A is the circle area size in the sketch image 90b drawn by the user B . Such a difference can be interpreted as a difference in taste or disposition of each individual user. Personal information can be used to acquire and analyze maps and other input information drawn by a user from a device and to grasp a sketch tendency of a specific user, a tendency to express distances, habits and the like.

Assuming that the size information of the circle representing the area of 1 km radius is accumulated and used repeatedly based on the learning from the sketch image 90a by the user A, when the user uses the circle, It is possible to guess how much information is desired to be searched for in consideration of the information (tendency information) accumulated through learning while considering the size of the area.

1 is a diagram for explaining the structure and concept of an information processing system according to an embodiment of the present invention;

2 is a diagram showing the configuration of an information processing system according to an embodiment of the present invention

3 is a diagram showing the configuration of an information recognition section in an information processing system according to an embodiment of the present invention

4 is a diagram illustrating an example of a schedule management information processing method in an information processing system according to an embodiment of the present invention;

5 is a diagram illustrating an example of a telephone number information processing method in an information processing system according to an embodiment of the present invention

6 is a diagram showing an example of an information processing method for image retrieval in an information processing system according to an embodiment of the present invention

7 is a diagram illustrating an example of a map-based area information processing method in an information processing system according to an embodiment of the present invention

8 is a diagram showing an example of a method for grasping a user's input intention in an information processing system according to an embodiment of the present invention

9 is a diagram showing an example of a method of grasping user's personal information in an information processing system according to an embodiment of the present invention

Claims (15)

An input device for inputting information handwritten by a user; A recognizing device for recognizing at least one of a character, a number, a symbol and a figure from the information and analyzing a relationship between at least any one of the characters, numbers, symbols, and figures to recognize meaningful contents; And And a control device for controlling execution of any one of a plurality of applications and providing the meaningful contents to the application, The plurality of applications including a phone book management application, The recognizing device distinguishes the name and the telephone number from the information, Wherein the control device provides the name and the telephone number to the telephone directory management application. The information processing apparatus according to claim 1, wherein the input device receives a sketch or graffiti image written by a user. delete The information processing apparatus according to claim 1, wherein the recognizing device constructs and recognizes user's personal information based on learning of a handwriting pattern. The information processing apparatus according to claim 1, wherein the plurality of applications include a schedule management application, and the meaningful contents include date and schedule contents. delete The information processing apparatus according to claim 1, wherein the plurality of applications include any one of an information retrieval application in an apparatus and an information retrieval application from an external apparatus based on wired or wireless communication. The information processing apparatus according to claim 1, wherein the plurality of applications include at least one of an image and a map / local information search application. Recognizing at least one of letters, numbers, figures, and symbols from contents written directly by a user in the form of a sketch or a graffiti using an input device; Analyzing a relationship between at least one of the characters, numbers, figures, and symbols to recognize meaningful contents; And  Executing one of a plurality of applications and providing the meaningful contents to the application, The plurality of applications including a phone book management application, Recognizing at least one of letters, numbers, figures, and symbols from the directly handwritten content  Recognizing the name and the telephone number separately from the written contents, The step of executing any one of a plurality of applications and providing the meaningful contents to the application And providing said name and said telephone number to said telephone book management application. The method as claimed in claim 9, wherein the step of executing any one of the plurality of applications and providing the meaningful contents to the application And providing the date and the word or sentence to the schedule management application when the meaningful content includes a date and a word or a sentence. delete The method as claimed in claim 9, wherein the step of executing any one of the plurality of applications and providing the meaningful contents to the application And providing the map / area information to the application when the meaningful content includes the map / area information. 10. The information processing method according to claim 9, wherein when the written content contains a specific symbol, the user's intention is grasped based on the symbol. 14. The method as claimed in claim 13, wherein the step of executing any one of the plurality of applications and providing the meaningful contents to the application And when the specific symbol is a symbol representing a query, searching the information based on wired or wireless communication as an application. The method as claimed in claim 9, wherein the step of executing any one of the plurality of applications and providing the meaningful contents to the application And performs information search such as an image or a web page based on wired or wireless communication based on the meaningful contents.

Images