KR19980042731A - Information input method and device - Google Patents

Abstract

There is provided an information input method and interface that can be easily operated without knowing the hardware directly. Image recognition is performed on the image information obtained from the target pattern on the object surface and the access instruction pattern generated by the user using the image recognition unit. The input pattern is generated using the image pickup section. The target pattern is associated with the corresponding information stored in the storage. By performing image recognition on the access instruction pattern, an access instruction input is specified that specifies a subset of the information. The control operation corresponding to the access instruction input is performed by the controller in response to the access instruction input.

Description

Information input method and device

The present invention relates to a method and apparatus for inputting data and / or control information into an electronic device. In particular, the present invention relates to systems and methods that use image recognition to input information into a computer or other data and / or control system.

As a device that combines the functions of existing audio-visual (AV) devices, information devices, and communication devices, various new multimedia devices that enable input / output of multimedia information have been proposed. Examples of such new devices include portable information terminals with communication functions, camcorders with communication functions, and personal computers with communication functions and AV functions. In addition, systems are being developed in which these new multimedia devices are designed to communicate with each other via different types of networks. Several different types of networks in use for internal device communications include Ethernet local area networks (LANs), token ring LANs, wired area networks (WANs), wireline networks such as public telephone networks, and infrared communication systems, Wireless communication networks such as cellular telephone systems and satellite communication systems.

Many of these new multimedia devices use conventional interfaces for information input, such as existing AV devices, information devices and communication devices. For example, conventional and new multimedia devices use keyboards and mice, touch panels, dedicated controllers and microphones.

As shown in FIG. 15A, a barcode is another example of conventional means used to enter information into a multimedia system. Barcodes typically contain line patterns that vary in thickness representing a binary code. Barcodes are typically used for machine identification of products and other items or to specify objects. Bar codes can be one-dimensional or two-dimensional. That is, the barcodes are arranged to be scanned by the barcode reader in one or two different scanning directions. 15B shows a two-dimensional bar code.

Unfortunately, the manipulation of conventional input interfaces is inconvenient and unsuitable for human emotion. Conventional devices themselves are not ergonomic, their operation is not intuitive, and it cannot be said that specific training and exercises are usually required before using them efficiently. For example, a keyboard may have many kinds of adjustments and certain comfortable features, but even the most ergonomic keyboard is difficult to use efficiently until the user has learned to practice touch typing. The same applies to a mouse, a touch panel, a dedicated controller, and a microphone used in a conventional multimedia device. Such input devices are very unsuitable for human emotion. That is, a user unfamiliar with operating such a device does not intuitively understand the related operation method and requires expert knowledge and training to be skilled in operating these devices.

In addition to being not easily recognized or read by humans, another problem arises that systems using barcodes have only a limited number of codes for a given bar pattern display area. The more different articles that need to be identified by such a system, the more unique patterns are required. Consequently, the number of bar code patterns must be increased or the number of articles that can be identified must be limited. If the number of patterns is increased, it is necessary to increase the bar pattern display area on the article. This is not a good solution. For example, in the case of a product, the surface area of the article is limited and is typically used to display other information. Moreover, adding more unique patterns by adding additional bars, each of which only adds two bits of information, increases the time it takes to recognize the bar code.

Accordingly, it is an object of the present invention to provide an information input device and an information input method that can be operated easily, efficiently and intuitively without having to learn or fully know how to use a particular hardware.

It is still another object of the present invention to provide an information input device and an information input method that enable a user to specify, relate, and refer to specific information from a large amount of data through a simple operation.

The above and other objects of the present invention can be achieved by the information input method and apparatus of the present invention. An information input method according to the present invention comprises the steps of recognizing a predetermined target pattern on an object surface, associating a recognized pattern with information, and an access instruction input signal generated by image recognition of an access instruction input pattern and corresponding to the associated information. And receiving a control operation corresponding to the access instruction input signal.

The information input apparatus according to the present invention includes image pickup means for picking up a target pattern and an access instruction pattern, image recognition means for recognizing a target pattern and an access instruction pattern picked up by the image pickup means, and information related to the target pattern. Storage means for storing, authentication processing means for associating a target pattern recognized by the image recognition means with information, and control means for receiving an access instruction input signal based on the information and performing a control operation based on the access instruction input signal; Include.

Now, with reference to the following detailed description of the invention and the accompanying drawings, these and other features of the present invention will be understood.

1 is a functional block diagram illustrating an embodiment of an information input apparatus according to the present invention.

2 is a block diagram illustrating a structure of a multimedia system to which the information input device of FIG. 1 is applied.

3 is a block diagram showing an embodiment of a hardware structure of a part of the information input device of FIG.

4 is a plan view showing a configuration example of an embodiment of the information input device of FIG. 1;

5A illustrates an example of code value assignment of a rotation icon code that can be used in the input sheet of FIG.

FIG. 5B shows a first example of a rotation icon code that can be used in the input sheet of FIG. 4. FIG.

6 is a plan view showing an embodiment of the input sheet of FIG. 4 for controlling an electronic apparatus having conventional video cassette recorder control.

FIG. 7 shows an exemplary output display of an output of an embodiment of the information input device according to the invention using the input sheet of FIG. 6; FIG.

Fig. 8 is a flowchart showing one embodiment of a recognition processing method for recognizing an object using an image recognition unit of an embodiment of an information input apparatus according to the present invention.

Fig. 9 is a diagram showing the amount of movement detected during position correction by the image recognition unit of the embodiment of the information input device according to the present invention.

Fig. 10 is a diagram showing a center of mass and an circumscribed rectangle of each icon extracted by a rotated icon extraction process in one embodiment of a recognition processing method for recognizing an object according to the present invention.

Fig. 11 is a diagram showing a selected rotation icon code group portion used in the rotated icon code determination processing in one embodiment of a recognition processing method for recognizing an object according to the present invention.

12 is a flowchart showing one embodiment of an input / selection instruction processing method in one embodiment of the image recognition unit of the embodiment of the information input apparatus of the present invention.

Fig. 13 is a diagram showing an access instruction input pattern for providing an input / selection instruction extracted by the input / selection instruction extraction process performed by the image recognition unit of the embodiment of the information input apparatus of the present invention.

14 shows a second embodiment of a rotation icon shown on the input sheet of FIG.

15A illustrates a different format of one-dimensional bar code used in the prior art input system.

15b illustrates a two-dimensional bar code used in a prior art input system.

Explanation of symbols for the main parts of the drawings

100: information input device

101: image pickup unit

102: image recognition unit

103: control unit

104: storage unit

105: output unit

107: external device

108: communication link

109: communication line

Referring to FIG. 1, a preferred embodiment of the information input device 100 according to the present invention includes an image pickup unit 101, an image recognition unit 102, a control unit 103, a storage unit 104, and an output unit 105. ). The control unit 103 is connected to the information distribution unit 106 via the communication link 108 and to the external device 107 via the communication line 109. The communication link 108 may be a network, for example, and the communication line 109 may be a home bus, for example.

FIG. 2 illustrates an application of the information input device 100 of FIG. 1. That is, Figure 2 illustrates the invention, for example, for use in a multimedia system. 1 and 2, the image pickup section 101 picks up an object to be recognized pattern added to a surface of an object such as paper and an access instruction pattern manufactured by a user with a finger or other similar pointer. In particular, the image pickup unit 101 is preferably a charge-coupled-device (CCD) video camera 201 or a quantum well infrared photodetector (QWIP) as shown in FIG. 2. ) It is realized with sensitive digital cameras such as video cameras. The CCD video camera 201 is preferably arranged on top of the viewer 205A as in the multimedia system of FIG. The CCD video camera 201 collects the captured image of the input sheet 209 set on the medium base 208. The medium base 208 is adjacent to the viewer 205A disposed in a vertical position such that the CCD video camera 201 located above the viewer 205A is focused on the input sheet 209 on the medium base 208. have.

In such an embodiment, the input sheet 209 serves as an information selection medium that is divided into different category areas. The input selection sheet 209 may be made of paper or thin plastic, for example. Referring to FIG. 4, an embodiment of the configuration of the input sheet 209 of FIG. 2 is shown. The embodiment shown in FIG. 4 shows an example input sheet 209 that includes five different category regions labeled A, B, C, D, and E. FIG. The particular category includes a number of separate areas on the input sheet 209.

The first category area A of the input sheet 209 is provided with an authentication icon that can be used to authenticate that the particular plastic or paper is the input sheet 209 that can be used in the information input device 100 of the present invention. In other words, if the input sheet 209 includes an authentication icon in the category area A that the information input device 100 can authenticate, the system enables operation with the specific input sheet 209. Otherwise, the system becomes inoperable with the particular input sheet 209. The combination of the authentication icons arranged in the predetermined category area is called a recognition target pattern.

In the second category area B of the preferred embodiment of the input sheet 209 is an identification icon (or icons) that associates the particular sheet with electronic information stored or accessible. The identification icon is also referred to as the recognition target pattern. In the third category area C, an input sheet 209 position correction mark is provided. The fourth category area D is provided with a user graphic for displaying a selection. Finally, the fifth category E is provided in the form of a list of information contents of the system. Hereinafter, the structure of the input sheet 209 provided and the configuration of the graphic will be described in more detail.

The image recognition unit 102 of FIG. 1 performs image recognition on the image information provided by the image pickup unit 101. In the embodiment of Fig. 2, the CCD video camera 201 photographs the input sheet 209 to generate image information. The image recognition unit 102 is implemented in the control box 202 in the multimedia system shown in FIG. The control box 202 with the image recognition unit 102 receives the image information output from the CCD video camera 201. The image recognition unit 102 performs identification and authentication processing of the recognition target pattern included in the image information received from the CCD video camera 201. Based on the recognition and authentication process, the image recognition unit 102 outputs the recognition result and the authentication result to the control unit 103.

As will be described later, once an association is made between the input sheet 209 and the electronic information as a recognition result of the recognition target pattern, the image recognition unit 102 recognizes the input / selection instruction made by the user and then recognizes the recognition result signal. Is output to the control unit 103.

Along the image recognition unit 102, the control unit 103 is provided in the control box 202 of the multimedia system shown in FIG. The controller 103 controls the transmission and reception of information between various other components. Based on the recognition result of the recognition target pattern by the image recognition unit 102, the control unit 103 determines whether the recognition target pattern is related to the information stored locally in the storage unit 104. If relevant, the controller 103 accesses locally stored related information.

If the recognition target pattern is associated with information that is not stored locally, the recognition result is sent to the information distributing unit 106. In this case, the related information is stored in the information distributing unit 106 or an information processing unit disposed on an external communication network connected to the information distributing unit 106. The information distributing unit 106 accesses remotely stored related information.

In any case, once the relevant information has been located, the controller 103 sends a copy of the related information to a temporary storage unit (not shown) provided within the controller 103. The relevant information preferably includes several types of data. For example, the information related to the input sheet 209 preferably includes display information for providing user feedback to the output unit 105, control information for controlling the external device 107, and a link for positioning other related information. Information and command configuration information for interpreting and using the input sheet 209.

Once the control unit 103 has a copy of the relevant information in the temporary storage unit, authentication is performed for each input / selection indicating image received from the image recognition unit 102. When the input / selection indication image is authenticated, the display screen on the output unit 105 is updated and the control function of the external device 107 is performed. That is, the display and the information distributing result indicating the function of these different operations are displayed on the output unit 105 when it is performed.

The forms of information listed above relating to different input sheets 209 or to different areas of the input sheet 209 will now be described in detail. The display information includes text, images, layout information, sound information, and the like. Based on this display information, the controller 103 generates a display on the output 105 and controls the sound generation.

The control information includes a control command for controlling the external device 107 in response to a user making an input / selection instruction on the input sheet 209. The controller 103 controls the external device 107 based on the control information associated with the input sheet 209. For example, if the external device 107 is a video cassette recorder (VCR), the control information includes play (play), stop, fast forward, rewind, and pause control information for controlling the operation of the VCR. For example, if the user points to a graphic on the input sheet 209 indicating the reproduction control information, the image pickup unit 101 recognizes the reproduction input / selection instruction image in response and transmits a reproduction command to the VCR. The image information is transmitted to 102.

The link information includes an address for reading another information when the user makes an input / selection instruction. The link may point to information stored in storage 104, information distribution 106 or an external information processor connected to a communication network. Based on the link information, the control 103 accesses further additional information. For example, as used on the Internet, a Uniform Resource Locator (URL) may be used as link information in the present invention.

The command configuration information includes information indicating what function will be performed when the graphic in the D or E category area of the information input sheet 209 is selected by the user. The process of the user selecting a graphic on the input sheet 209 is also referred to herein as a user input / selection instruction. The instruction configuration information is in the form of a physical position on the input sheet 209 when selected by the user and the next processing instruction corresponding to the selected position. For example, the command configuration information may be a processing instruction for executing the above-described control information or link information when the position coordinate and the appropriate position coordinate are selected by the user.

In addition to the control unit 103 and the image recognition unit 102, the storage unit 104 is provided in the control box 202 in the multimedia system shown in FIG. The storage unit 104 stores information related to the recognition target pattern. The information stored in the storage unit 104 includes information for controlling the external device 107. Such information may be retrieved from an external information processor connected to the network via the information distributing unit 106 or from a remote information source via, for example, a satellite communication system. By placing the storage 104 in the control box 202, the amount of transmission on the network may be reduced and the response time for user input / selection instructions may be shortened.

The output 105 preferably comprises a display, a speaker and / or a printer. Accordingly, the output unit 105 may provide a sound and visual indication of the function of the operation selected by the user and the information distribution result. 7 is an example of outputting sample information to the output unit 105 display screen. On the left side of the display screen 700 is a graphic 710 showing the function of the action selected by the user. Note that the VCR control appears and the play command graphic 7200 is dark. This represents the user selection of the playback command. On the right side of the playback screen 700, an image 730 output from the external device 107 indicating the information distribution result is shown. Image outputs of various other devices under control of the system may be synthesized and output to the output unit 105 shown in FIG. The output 105 of the multimedia system shown in FIG. 2 is implemented as a viewer 205A and a video scan converter 205B.

The information distribution unit 106 may be connected to the control unit 103 through a wired communication network such as an Ethernet LAN, a token ring LAN, or an ATM WAN. Alternatively, wireless communications networks such as infrared, PHS telephones, digital cellular telephones or satellite communications systems may be used.

If the control unit 103 determines that the recognition result from the image recognition unit 102 requires information related to the specific input sheet 209 that is not available locally in the storage unit 104, the information distribution unit 106 determines the communication network. The relevant information is retrieved from an external information processing unit connected to the system through. In the multimedia system shown in FIG. 2, the information distributing unit 106 is implemented as a programmed workstation 206.

The external device 107 can be any number of different electronic devices that can be controlled by the controller 103 in response to an input / selection indication. For example, in FIG. 2, the external device 107 includes an AV controller 207A, a VCR 208B, a modem 207C, a telephone 207D, and a camera 207E. The external device 107 may include various types of electronic devices such as a computer and a game machine. The external device 107 may be connected to the controller 103 via a wired communication line or a wireless communication system.

3 illustrates an exemplary hardware embodiment of the control box 202 of FIG. 2. The control box 202 of FIG. 2 includes a central processing unit (CPU 301), an image processing unit 302, a camera interface 303, a display interface 304, a network interface 305, a home network interface 306, and a readout. A dedicated memory [ROM 307], a random access memory [RAM 308], and a hard disk drive [HDD 309]. All components of the control box 202 may be connected to each other via the bus 310. Alternatively, these components can be connected to each other via a dedicated communication path. For example, a camera interface 303 is shown that is directly connected to the image processing unit 302 by a dedicated link. The CPU may also include a second separate bus for connection with the ROM 307 and / or the RAM 308.

The CPU 301 is suitable for controlling the control box 202 according to a system program stored in the ROM 307 or a program developed in the RAM 308. The CPU 301 provides some functions of the image recognition unit 102 and some functions of the control unit 103 shown in FIG. That is, the image recognition unit 102 and the control unit 103 may be partially implemented as the CPU 301.

Programs and data necessary for the CPU 301 to perform various processes are stored in the ROM 307. The ROM 307 provides some functions of the control unit 103 of FIG. The RAM 308 is suitable for developing and temporarily storing programs and data necessary for the CPU 301 to perform various processes. RAM 308 also provides some functionality of control 103 in FIG. That is, the controller 103 may be partially implemented as the ROM 307 and the RAM 308 together with the CPU 301. Since HDD 309 provides the functionality of storage 104 of FIG. 1, storage 104 may be implemented as HDD 309.

The image processing unit 302 receives image information from the CCD video camera 201 through the camera interface 303. Various image processing such as image recognition processing is performed by the image processing unit 302. The image processing unit 302 provides some functions of the image recognition unit 102 of FIG. 1. The camera interface 303 receives image information from the CCD video camera 201 and then converts the image information into a signal format compatible with the image processing unit 302. After that, the camera interface 303 outputs the converted image information to the image processing unit 302. Therefore, the image recognition unit 102 may be partially implemented as the camera interface 303 and the image processing unit 302 together with the CPU 301.

The display interface 304 receives the display data processed by the CPU 301 and the image processing unit 302, converts the display data into a signal compatible with the viewer 205A, and then converts the converted signal into the viewer 205A. Output to Display interface 304 and viewer 205A provide some functionality of output 105 of FIG. 1. Accordingly, the output unit 105 may be partially implemented as the display interface 304 and the viewer 205A.

The network interface 305 provides a connection to the workstation 206 to enable access to an external network. Network interface 305 and workstation 206 provide some functionality of communication link 108 and information distributor 106 of FIG. The communication link 108 and the information distribution unit 106 may be partially implemented as the network interface 305 and the workstation 206.

The home network interface 306 provides a connection to the external device 107. Thus, the CPU 301 can control the external units 207A-207E through the home network interface 306. The home network interface 306 and external units 207A-207E provide some functionality of the communication line 109 and external device 107 of FIG. 1. The communication line 109 and the external device 107 may be partially implemented as a home network interface 306 and a plurality of external units 207A-207E such as a VCR, modem, video camera, and the like.

The above-described components 301-309 are connected to each other via the bus 310. In addition, these components provide various functions of the information input apparatus as described above with reference to FIG.

As described above with reference to FIGS. 1 to 3, the input sheet 209 captured by the information input apparatus with reference to FIG. 4 will be described in more detail below. As described above, the input sheet 209 is preferably divided into five category areas A, B, C, D and E.

In the first category area A of the input sheet 209, an authentication icon is provided for authenticating that a particular plastic or paper is an input sheet 209 that can be used in the information input device 100 of the present invention. As described with reference to Figs. 1, 2 and 3, an authentication icon is provided as a recognition target pattern. The authentication icon in the category area A is compared with the comparison icon information captured by the CCD video camera 201 and stored in the ROM 307. If the CPU 301 finds a match between the authentication icon of the input sheet 209 and the comparison icon information stored in the ROM 307, the operation proceeds to the authentication process using the category area B. If it does not match, an error message is displayed on the viewer 205A.

In the second category area B, an identification icon (or icons) for associating a specific sheet with electronic information is provided as a recognition target pattern. The identification icon is directional and has a certain number of discernable orientations. For example, in the embodiment of the input sheet 209 in FIG. 4, an icon similar to a key was used. The key shape has a narrow end and a thigh end to provide directionality and to identify its orientation. Moreover, different orientations of the identification icons represent different values, codes or meanings. In the embodiment shown in Fig. 4, each identification icon serving as a recognition target pattern portion in category area B has eight distinct orientations. Thus, each identification icon can be oriented such that six separate code values can be represented based only on the orientation of the icon. 5A shows an example of eight distinct orientations of an icon in the form of a key having one of eight separate code values assigned to each orientation.

By using only a few additional clearly oriented icons together in a sequence, many more distinct codes can be represented. For example, using a sequence of four identifying icons, each with eight distinct orientations, 4096 different code values can be represented. The number of values that can be represented by four icons with eight orientations is calculated as follows.

(8 orientations) _{icon 1} * (8 orientations) _{icon 2} * (8 orientations) _{icon 3} * (8 orientations) _{icon 4} = 8 ⁴ = 4096 values

Using the example orientation value assignment shown in FIG. 5A, the sequential sequence of identification icon 500 shown in FIG. 5B represents a code value of 2574. That is, the first key icon 510 represents two, the second key icon 520 represents four, the third key icon 530 represents seven, and the fourth key icon 540 represents four.

In the third category area C, a position correction mark is provided. The position correction mark is used to perform position correction of the image picked up by the CCD video camera 201.

In a fourth category area D, graphics are provided for the user to perform selection instructions. By selecting the graphic in the area D, the user can select the information displayed in the viewer 205A but not in the category area E of the input sheet 209. That is, the category area D is used to make a selection instruction whenever the selection information provided in the category area E of the input sheet 209 differs from the selection information displayed on the viewer 205A. This allows the information input device 100 to dynamically add, via the viewer 205A, an input selection option that does not exist in the category area E. FIG.

For example, as shown in FIG. 4, if up, down, left and right arrow graphics are provided in category area D, the user may move the cursor and / or select a menu selection on the menu screen displayed on viewer 205A. To perform the determination, any arrow graphic can be selected in the category area D. Although the graphics provided in category area D can be implemented with up, down, left and / or right arrow pictures to provide cursor movement functions, the graphics need not be limited to such arrow shaped pictures and the functions are such cursor movements. There is no need to be limited to functionality.

For example, when the input sheet is used as a common medium for controlling the VCR and the audio system, graphics are provided to control the functions common to external devices. VCRs and audio systems often have common functions such as rewind, stop, and record functions. Thus, by associating the graphics with the common configuration information including the mounted processing instruction and the positional information in the category area D, execution of the mounted processing instruction is possible. In the above-described example, the mounted processing instructions include up, down, left and right cursor movement instructions and playback, stop and record instructions. However, for a CD player, for example, similar graphics may be used for commands such as going forward in a track, going forward in a next track, going backward in a track, going backward in a previous track, ejecting, stopping, and changing the information display mode. have. Note that once the command configuration information is copied to the temporary storage in the control section 103, the information sheet 209 is authenticated and identified.

Substantial information content available through the information input device 100 is provided in the fifth category area E. FIG. This list of contents can be displayed in a number of ways. For example, an image map similar to a hyper-text transmission protocol (HTTP) image map used in the form of a table describing general topics arranged in object reference order or used for World Wide Web (WWW) pages on the Internet. And as detailed indexes each itemizing every bit of substantial information available in alphabetical order. By looking at the information described in this category area E, the user learns what information can access the particular input sheet 209 recently captured by the CCD video camera 201.

In addition, once the specific input sheet 209 is recognized by the information input device 100 shown in Fig. 1, the user can select from the content information provided in the category area E by simply pointing out a desired item of information in the content information. It may be. The pointing step provides a display selection picture pattern that can be recognized by the interface system 100. Thereafter, after the indication of the specific selection is provided to the controller 103, some predetermined processing can be performed.

As an example, consider the input sheet 209 of FIG. The category area E includes a graphic image map of the VCR control button. Graphic 600 representing six different control buttons is intuitive for the VCR functions of Play 610, Stop 620, Pause 630, Rewind 640, Fast Forward 650, and Takeout 660. Provide a set of controls. This exemplary arrangement allows the user to easily operate the VCR using six control buttons. Although the information content described in the category area E has been described using the control buttons for the VCR, those skilled in the art will appreciate that many other devices can be controlled using the appropriate input sheet 209 and, in particular, that the information content can be controlled. It can be appreciated that such a VCR control button is not limited.

As another example, the content information in the category area E may be in the form of a series of menus. Thus, in response to the user pointing out a particular menu in the category area E using his or her finger, the next display of the menu item corresponding to the selected menu is displayed on the viewer 205A. Therefore, by linking the menu in the category area E with the link information and the command configuration information including the mounted processing command and the positional information, it is possible to display the link information and execute the mounted processing command. In this embodiment, the next list of menu items is displayed on viewer 205A in response to the user pointing to the location of the relevant initial menu on pip sheet 209. Once the relevant link information and command configuration information are copied to the temporary storage in the control section 103, the information sheet 209 is authenticated and identified.

In the multi-media system constructed as described above, the image recognition unit 102 executes an identification process for identifying the form of the input sheet 209 to be used. Once this occurs, the input sheet 209 is authenticated by the authentication process of the image of the input sheet 209 received from the CCD video camera 201.

The authentication process is performed by first performing preprocessing such as noise removal, variable density processing and threshold adjustment. Position correction is then performed. The system then proceeds to extract the center of mass and the trajectory of the authentication icon used as the recognition object pattern in the category area A in the upper center of the input sheet 209. Finally, the system attempts to match the image information with the stored authentication icon pattern.

Next, an authentication process occurs. For example, as shown in the flowchart of FIG. 8, noise reduction, variable density processing, and threshold adjustment for image information obtained when imaging the output of the CCD video camera 201 condensed on the input sheet 209. Pre-processing as described above (S801) is first performed. Then, the position correction process S802 is preferably performed by first detecting and measuring the positional shift based on the image information of the position correction mark provided in the category region C located at four corners of the input sheet 209. . Thereafter, the position correction process S802 performs transformation of the coordinate system in correspondence with the position movement amount. Then, the extraction operation S803 of the identification icon provided as the recognition target pattern in the category area B of the input sheet 209 is performed, and the determination processing S804 of the rotated icon code is performed.

In the position correction processing (S802), the position movement at the reference position is detected based on the image information obtained when imaging the output of the CCD video camera 201 focused on the category area C of the input sheet 209. As shown in Fig. 9, the system overlays the correction pattern on the position correction mark and detects the direction and magnitude of the predetermined position error. For example, if the input sheet is properly aligned, a match 900 occurs, and if the input sheet 209 is too high and left off, a shift to the upper left 910 occurs, and the input sheet 209 Too high and off to the right causes a shift to the upper right 920. If the coincidence 900 pattern is not detected, position correction is performed by mathematically converting the coordinate system of image recognition according to the direction and magnitude of position movement.

In the extraction process of the identification icon S803, as shown in FIG. 10, from each image icon obtained from image information obtained when capturing the output of the CCD video camera 201 focused on the category area D of the input sheet 209, the respective icons are displayed. The center of mass and circumscribed rectangle of variable density values are found. In Fig. 10, the center of mass of each icon determined by the identification icon extraction process (S803) is indicated by a white spot and the circumscribed rectangle is indicated by a broken line.

In the rotation icon code determination process S804, eight different orientations of the icon are determined with reference to the direction characteristic table of the rotation icon. The direction characteristic table defines eight different orientations with respect to two parameters, the aspect ratio of the circumscribed rectangle found in the center of mass and the identified icon extraction process S803. The direction characteristic table is represented by Table 1 below. It is selected based on the conditions indicated by the characteristics of each icon orientation. For example, the second row of Table 1 shows that if the aspect ratio of the circumscribed rectangle is at least 2.5 and the center of mass is located on top of the circumscribed rectangle, the rotation icon is oriented downward. Based on the orientation assignment of the icon with the code value as shown in FIG. 5A, the rotation icon is converted to a constant code. This process is repeated for each of the four icons to determine the four code sequences.

Icon orientation characteristic lift Aspect ratio 5: 2, center of mass located at the bottom of the rectangle Downward Aspect ratio 5: 2, center of mass located on top of the rectangle Right Aspect ratio 2: 5, center of mass located on the left side of the rectangle Left Aspect ratio 2: 5, center of mass located on the right side of the rectangle Upper right Aspect ratio 1: 1, center of mass located in the lower left area of the rectangle Lower right Aspect ratio 1: 1, center of mass centered on top left area of the rectangle Upper left Aspect ratio 1: 1, center of mass located in the bottom right area of the rectangle Lower left Aspect ratio 1: 1, center of mass centered on top right area of the rectangle

Hereinafter, the rotation icon code sequence found from the four rotation icons is compared with the selected rotation icon code group as shown in FIG. The best match between the rotated icon code sequence and one of the selected code groups is selected as the rotated icon code of the input sheet 209. The best match is determined by finding the selected rotation icon code group having the minimum sum of differences between the selected rotation icon code group and each portion of the rotated icon code sequence. If the sum of all the selected rotation icon code groups is greater than or equal to the threshold, the object to be recognized is not recognized as a known code and is determined to be unrecognizable.

As described above, the controller 103 determines whether an association can be made between the recognition result of the recognition object by the image recognition unit 102 and some information stored in the storage unit 104. If relevant, the information is accessed. In other words, if association with information stored in the device itself can be made, the relevant information is immediately accessed. If there is no information related to the recognition result from the image recognition unit 102 currently stored in the storage unit 104, the recognition result is transmitted to the information distribution unit 106.

In this case, the related information is stored in the information distributing unit 106 or the information processing unit existing in the external communication network connected to the information distributing unit 106. Therefore, after the information relating to the recognition result of the object to be recognized by the image recognizing unit 102 is arranged by the information distributing unit 106, the control unit 103 reads the related information into the temporary storage unit provided to the control unit 103. .

Thus, when the identification of the input sheet 209 and the determination of this type are completed, the processing is performed based on the information read into the temporary storage provided in the control section 103. All of these occur in response to an input / selection instruction from the image recognition unit 102.

FIG. 12 shows an embodiment of the steps used for the input / selection instruction processing by the image recognition unit 102. As shown in FIG. Preprocessing S1201 is performed first. This process includes functions such as noise cancellation, variable density processing and threshold adjustment. The preprocessing S1201 is performed on the image information obtained when imaging the output of the CCD video camera 201 during the input / selection instruction made on the input sheet 209 made by the user.

Thereafter, the position correction processing (S1202) starts by detecting the position movement as shown in FIG. 9 based on the image information of the position correction mark provided in the category region C located at four corners of the input sheet 209. do. The position correction process S1202 proceeds by calculating a mathematical transformation of the coordinate system in correspondence with the magnitude and direction of the position movement as described above with reference to FIG. As a result, position-corrected binary image information is obtained.

In the next step, the extraction processing of the input / selection instruction S1203 is performed and then the determination processing of the input / selection instruction S1204 is performed.

In the input / selection instruction extraction process S1203, the already stored unique image of the input sheet 209 is compared with the variable density image from which the input / selection instruction is detected. For example, as shown in Fig. 13, position-corrected variable density image information, which may be a pattern indicated by a finger, is extracted as a differential image from the unique image of the input sheet 209. That is, an access instruction input pattern is determined to specify an input / select instruction. The display area includes the differential image extracted as a whole. Thus, the pixel distribution in the circumscribed rectangle and the display area can be determined.

Then, in the input / select instruction determination processing S1204, the direction of entry of the pointer into the display area is determined from the pixel density distribution on each edge of the circumscribed rectangle. Image scanning in the rectangle is performed in relation to the entry direction and the end point position of the pointer is determined. In the example embodiment of Fig. 11, the direction of entry of the pointer is found by confirming that a high density of pixels have been distributed to a particular edge of the display area. The category area E is divided into blocks of a plurality of known areas, and image scanning is performed by scanning for pixels in these blocks based on the pointer entry direction and corresponding scanning pattern specified in Table 2. Thus, by scanning only the display area of the image divided into blocks, high speed detection of the pointer end point in the limited scanning area is possible.

Direction of entry Scanning pattern From right edge Scan the left end block of the rectangle from the top edge Enter from left edge Scan the right end block of the rectangle from the top edge Enter from top edge Scan the lower end block of the rectangle from the left edge Enter from bottom edge Scan the upper end block of the rectangle from the left edge Enter from top right edge Scan the lower left end block of the rectangle from the left edge Enter from lower right edge Scan the upper left end block of the rectangle from the left edge Enter from top left edge Scan the lower right end block of the rectangle from the left edge Enter from lower left edge Scan the upper right end block of the rectangle from the left edge

The detected end point positions are converted into corresponding commands / information and transmitted to the control unit 103. In the embodiment shown in Fig. 13, the pointed position of the finger is converted into the corresponding area number 6. In Fig. 13, the hatched area is a block in the category area E corresponding to the area number 6.

Finally, the controller 103 is defined in the command configuration information included in the retrieved related information of the identification input sheet 209 stored in the temporary storage of the controller 103.

Therefore, in the information input device 100, the authentication process is performed by image recognition of the recognition target pattern provided on the input sheet 209. That is, a static medium can be used as a recognition object, and a relationship can be made between the recognition object and the corresponding information. An access and display input that specifies related information is detected by image recognition of the access input pattern. Thus, dynamic access of information can be performed using the recognition object indicated on the static medium.

The pattern to be recognized is not limited to the pattern in the form of a key as in the above-described embodiment. It may be any unique pattern with directivity. Thus, for example, by combining the shape and orientation of another recognition pattern as shown in FIG. 14, a limited number of different types of input sheet 209 can be defined.

Further, after recognition of the input sheet 209, the input / selection instruction may be performed by simply pointing out the desired input / selection instruction content of the input sheet 209 by hand. Therefore, the information input / output / selection operation can be easily performed without knowing the hardware directly, and the input / selection instructions can be associated with specific information from a very large data capacity.

Further, in the above-described embodiment, the center of mass and the circumscribed rectangle of the variable density value of the image of the icon are used for the recognition of the rotated icon code. However, the orientation to the prepared template image can be found by the matching process.

As described above, in the information input apparatus and method according to the present invention, image recognition of the recognition target pattern displayed on the object surface is performed, and the optically recognized target pattern is associated with the corresponding information. Next, an access instruction input corresponding to the related information is received by image recognition of the access instruction input pattern. A control operation corresponding to the received access indication input is performed based on the information related to the recognition target pattern. Therefore, the user can easily perform the input / output / selection operation without knowing the hardware directly.

In addition, in the information input apparatus and method thereof according to the present invention, the control contents corresponding to the access instruction input and the information transmission result are displayed according to the information related to the recognition target pattern. In other words, the present invention uses a small display area to index a large amount of data and has a user-friendly interface. Therefore, the information input device of the present invention can be used to easily, efficiently and clearly specify specific information from a large amount of data.

Further, in the information input device and the method according to the present invention, the authentication pattern and the identification pattern are recognized as the recognition target pattern displayed on the surface of the object. Only after the authentication process is performed on the recognized authentication pattern, there is a recognized confirmation mark associated with the corresponding information. Therefore, illegal access instruction input is prevented, and only access instruction input by a legitimate user is retrieved for information.

Further, in the information input apparatus and the information input method according to the present invention, whenever an access instruction input is received by image recognition of an access instruction input pattern, it is performed in the recognized authentication pattern. Only when the pattern is authenticated will the control action corresponding to the access indication be performed. Therefore, illegal access instruction input is prevented, and only the access instruction input by a legitimate user performs the operation.

Therefore, the present invention provides an information input method and an information input apparatus that can be easily operated by a user without knowing the hardware directly. In addition, the present invention makes it possible to directly relate to specific information contained in a large amount of data in a small and simple interface. Simple operation allows the user to directly access the desired information.

Various other variations and modifications of the operation structure and method of the present invention will be apparent to those skilled in the art without departing from the scope and spirit of the present invention. Although the invention has been described in connection with specific preferred embodiments, the invention as claimed should not be unduly limited to such specific embodiments. It is intended that the following claims define the scope of the invention and that the structures and methods and equivalents thereof within the scope of the claims be defined by the claims.

Claims (7)

In the information input method, Performing image recognition of a target pattern on the object; Associating the recognized object pattern with information; Performing image recognition of an access instruction pattern on the object; Associating the recognized access indication with the subset of related information; And Performing a control action in response to the associated subset of related information. Information input method comprising a. The method of claim 1, Displaying a control operation confirmation corresponding to the recognized access instruction pattern; And Displaying output information corresponding to the performed control operation Information input method further comprises a. The method of claim 1, wherein performing image recognition of the target pattern on the object comprises: Performing image recognition of the authentication pattern; Performing an authentication process on the recognized authentication pattern; And Performing image recognition of the identification pattern Information input method comprising a. 4. The information input method according to claim 3, further comprising performing an authentication process on the recognized authentication pattern before performing the control operation. In the information input device, Image pickup means for picking up an object pattern and an access instruction pattern on the object; Image recognition means for performing image recognition of image information picked up by the image pickup means; Storage means for storing information associated with the target pattern; Processing means for associating the target pattern recognized by the image recognition means with corresponding information; And Control means for receiving an access instruction input from the image recognition means in response to recognizing the access instruction pattern, and performing a control operation corresponding to the access instruction input; Information input device comprising a. In the information input device, An image recognition circuit for recognizing a target pattern and an access instruction pattern; A data access circuit coupled to the image recognition circuit, for accessing specified operation information based on the recognized target pattern and the access instruction pattern; And A control circuit coupled to the data access circuit, the control circuit performing an operation in response to the operation information accessed by the data access circuit. Information input device comprising a. 7. An information input apparatus according to claim 6, further comprising an output display coupled to the control circuit, instructing the performance of the operation and displaying output information generated by the performance of the operation.