JP2009015720A - Authentication device and authentication method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an authentication device and an authentication method which have simple structure and recognize one optional object. <P>SOLUTION: A marker detection part 12 detects a marker 24 added to an object 23 for input/output which is an identification object and formed so that the marker may be distinguished from the object 23 for input/output. In addition, an object area detection part 13 detects an area of the object 23 for input/output synchronized with a motion of the detected marker and a feature point extraction part 22 extracts feature points such as size, the shape, etc. of the object 23 for input/output on the basis of the area detected by the object area detection part 13. Thus, the object 23 for input/output is recognized without registering in advance the object 23 for input/output and by simple processing. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a recognition apparatus and a recognition method for recognizing an object.

As a technique for identifying an arbitrary object present in the surroundings, a technique described in Patent Document 1 is known. This Patent Document 1 describes that an automobile traveling on a road surface and recognizing the positional relationship between the automobile and the road surface. Further, a technique described in Patent Document 2 is known as a technique for detecting only information that an object exists without specifying an object using a sensor. In addition, a technique for recognizing an object by extracting feature points of an arbitrary object in an image only by image processing is known (Non-Patent Document 1).
Japanese Patent No. 3424334 Japanese Patent No. 3731123 Shun Kimura, Shinichi Aratani, Yasuo Tsuji “3D shape recognition by color object extraction and feature point detection”, Proc. Of 8th Image Sensing Symposium, H8, pp. 381-384 (2002)

  By the way, in recent years, opportunities for inputting characters such as e-mails on mobile terminals such as mobile phones have increased. Generally, in a mobile phone, a desired character can be input by pressing a numeric keypad a plurality of times.

  On the other hand, the keyboard of such a portable terminal has a problem that it is difficult to hit because the buttons are small. Therefore, it is considered that such a problem can be solved if the user can select an object of a desired size from the surrounding objects and use it as an input means. It needs to be recognized as an object.

  However, in the technology for recognizing an object known as the above-mentioned conventional technology, it is very difficult to specify an arbitrary object around as the only object to be used. For example, in the technique described in Patent Document 1, it is possible to recognize a road surface, a car, and the like, but it is not possible to recognize only one arbitrary object, and recognizes an object that is close to the above-described user. It is inappropriate to apply to

  Moreover, since only the position is recognized in the technique described in Patent Document 2, it is impossible to recognize only one arbitrary object.

  Further, the technique described in Non-Patent Document 1 has a problem that its configuration becomes complicated because it requires advanced image processing.

  Therefore, in order to solve the above-described problems, an object of the present invention is to provide a recognition device and a recognition method that can recognize an arbitrary object with a simple configuration.

  In order to solve the above-described problems, a recognition apparatus according to the present invention detects a marker that is added to an object to be identified and is formed so as to be distinguishable from the object, and is detected by the marker detection means. Object region detecting means for detecting a region synchronized with the movement of the marker, and feature point extracting means for extracting feature points of the object based on the region detected by the object region detecting means.

  In addition, the recognition method of the present invention synchronizes with a marker detection step for detecting a marker that is added to an object to be identified and formed so as to be distinguishable from the object, and a movement of the marker detected by the marker detection step. An object area detecting step for detecting an area; and a feature point extracting step for extracting a feature point of the object based on the area detected by the object area detecting step.

  According to the present invention, a marker that is added to an object to be identified and formed so as to be distinguishable from the object is detected, a region that is synchronized with the movement of the detected marker is detected, and based on the detected region, A feature point of an object can be extracted. Thereby, an object can be recognized by simple processing without registering the object in advance.

  Moreover, it is preferable that the recognition apparatus of the present invention includes display means for displaying an image that matches the feature point of the object.

  According to the present invention, it is possible to display an image that matches a feature point of an object, and to realize an easy-to-use device that enlarges or reduces the image to be displayed.

  In addition, the recognition apparatus of the present invention includes a state calculation unit that calculates the state of the marker, and a conversion unit that processes an image displayed on the display unit according to the state of the marker calculated by the state calculation unit. The display means preferably displays the video processed by the conversion means.

  According to the present invention, an image can be processed according to the calculated marker state, and the processed image can be displayed. For example, processing such as enlargement, reduction, angle conversion, etc., can be performed according to the position, inclination, etc. of the marker. By performing the above, display according to the state of the object can be performed, and a more convenient device can be realized.

  The recognition apparatus of the present invention includes an input detection unit that detects a positional relationship between the object and an operation position by a user, and a reception unit that receives an input operation based on the operation position by the user detected by the input detection unit. Preferably, the display means displays an image constituting an operation screen according to the characteristics of the object, and the user can perform an operation according to the image constituting the operation screen.

  According to this invention, while displaying an operation screen, the operation position by the user with respect to an object can be detected, and input operation based on the object and the operation position can be received. Therefore, the user can perform an operation according to the video constituting the operation screen, and can handle an object arbitrarily selected by the user as an input unit, thereby realizing a user-friendly device. Therefore, in the case of a small key such as a cellular phone, when the user arbitrarily selects a relatively large object as the input means, it is possible to realize an easy-to-use input means.

  According to the present invention, an object can be recognized by simple processing without registering the object in advance.

  The present invention can be readily understood by considering the following detailed description with reference to the accompanying drawings shown for the embodiments. Subsequently, embodiments of the present invention will be described with reference to the accompanying drawings. Where possible, the same parts are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is a block diagram illustrating a functional configuration of the recognition apparatus 100 according to the present embodiment. The recognition apparatus 100 includes a detection unit 10, a processing unit 20, a conversion parameter calculation unit 30, a video storage unit 40, a conversion unit 50 (conversion unit), a display unit 60 (display unit), and an input detection unit 70 (input / output detection). Means, receiving means)). Here, the detection unit 10 includes a registration unit 11, a marker detection unit 12 (marker detection unit), and an object region detection unit 13 (object region detection unit). The processing unit 20 includes a marker state calculation unit 21 ( State calculation means) and a feature point extraction unit 22 (feature point extraction means).

  FIG. 2 is a hardware configuration diagram of the recognition apparatus 100. As shown in FIG. 2, the recognition device 100 physically includes a CPU 101, a RAM 102 and a ROM 103, which are main storage devices, a sensor 104 (corresponding to the marker detection unit 12 and the object region detection unit 13), and an auxiliary storage such as a hard disk. The computer system includes a device 105, a display unit 105 such as a display (corresponding to the display unit 60), a short-range wireless communication unit 106 (corresponding to the input / output detection unit 70), and the like. Each function described in FIG. 1 has a sensor 104, a display unit 105, and short-range wireless communication under the control of the CPU 101 by loading predetermined computer software on the hardware such as the CPU 101 and the RAM 102 shown in FIG. This is realized by operating the unit 106 and reading and writing data in the RAM 102. Hereinafter, each functional block will be described based on the functional blocks shown in FIG.

  The registration unit 11 registers a marker to be attached to an arbitrary object in advance, information indicating the characteristics of the marker, and initial values (shape, color, size, position, inclination, which are reference values based on the characteristics) Etc.) is stored. For example, a marker is an object on which a predetermined pattern is formed, an object having dynamic information such as light emission, or an object that can be sensed by a sensor. Registered and stored by the user.

  Note that this marker may be a marker prepared in advance, or a pattern of a picture written by the user or a pattern extracted from surrounding objects may be automatically recognized and registered. . The initial value may be specified by the user, or may be automatically registered when the power is turned on or registered. Further, the marker information registered in the registration unit 11 may be information for identifying the marker and information indicating the state (position, inclination).

  The marker detection unit 12 is a sensor that detects a marker, and detects only the marker from the surrounding space included in the detection range of the marker detection unit 12 based on information stored in the registration unit 11. . Any sensor such as a position sensor, an acceleration sensor, and a geomagnetic sensor can be used for the marker detection unit 12, and a camera can be considered as one that can be easily used. A marker is obtained by capturing a surrounding image with a visible light camera or an imaging device using infrared irradiation, performing image processing on the captured image, and comparing it with the feature of the marker registered in the registration unit 11 using a technique such as template matching. Can be detected. The marker detection unit 12 always follows the movement of the recognized marker, calculates the marker state, and notifies the object region detection unit 13 of the state.

  When a marker is detected by the marker detection unit 12, the object region detection unit 13 is a part that detects an aggregate having characteristics that are synchronized with the movement of the detected marker. Detect as output object. For example, the user arbitrarily selects an object he / she wants to use for input / output, and the user attaches a marker to the selected object in order to detect the arbitrary object as an input / output object. At this time, an initial value of the reference marker state, for example, any one or a combination of marker positions and inclinations may be registered in the registration unit 11. Then, the object area detection unit 13 detects an aggregate including features synchronized with the movement of the marker as an object area, and recognizes the aggregate as an input / output object.

  As described later, the object region detection method may use an optical flow to recognize a portion that moves in synchronization with the marker as an input / output object, or recognize an edge of the object region. The input / output object may be recognized, or portions having the same color may be recognized as the same input / output object using color information. Further, the object area detection unit 13 may recognize an area using two or more pieces of information. For example, the object area detection unit 13 may detect the entire area having the same color using color information in the vicinity where the same set of optical flows exists.

  FIG. 5 shows a concept of detection of an input / output object to which a marker using an optical flow or the like is added. As shown in FIG. 5, the object region detection unit 13 detects a portion synchronized with the movement of the marker 24 detected by the marker detection unit 12. For example, movements (portions indicated by arrows) in each portion such as the edge portion and the surface of the input / output object 23 are detected.

  The marker state calculation unit 21 uses the initial value stored in the registration unit 11 as a marker state detected by the marker detection unit 12, for example, any one or a combination of position (including distance from the user) and inclination. It is calculated based on the standard. As a calculation method, a marker detection method in AR (Augmented Reality) can be used.

  The feature point extraction unit 22 is a part that calculates feature points such as the shape and size of the input / output object 23 detected by the object region detection unit 13 and the coordinates constituting the feature point. For example, when the input / output object 23 is a rectangular paper, the feature point extracting unit 22 can calculate the size of the input / output object 23 by extracting two diagonal points. it can. Further, by extracting at least 2 points, preferably 4 points, into the input / output object 23 for the object having a complicated shape such as a doll or a stuffed toy, the input / output object 23 is not rectangular, It is possible to grasp the size at which an image can be projected onto the input / output object 23. These two points or four points may be manually designated by the user, or may be determined automatically using image processing. The feature point extraction unit 22 grasps at least the size of the input / output object 23, and the display unit 60 can display an image matched to the input / output object 23 according to the grasped size. .

  Based on the two-point or four-point extracted portion, the feature point extraction unit 22 approximates the feature point set region by a rectangle, or selects the outermost point of the set (having the same feature). By connecting and detecting the inside as a region, a portion where an image can be projected onto the input / output object 23 can be recognized.

  Further, when the object region detection unit 13 detects the entire region sharing the same color using color information in the vicinity where the same set of optical flows exists, the feature point extraction unit 22 uses the two Using the above features, feature points can be extracted. When feature points are extracted in this way, the feature point extraction unit 22 stores the positional relationship between the extracted feature points and markers.

  The conversion parameter calculation unit 30 is a part that determines the state of the marker 24 (that is, the state of the input / output object 23) and calculates a conversion parameter for converting the video displayed on the input / output object 23. Since the state of the marker 24 and the input / output object 23 is the same, the state of the input / output object 23 (position (distance from the user), inclination, etc.) from the state of the marker 24 calculated by the marker state calculation unit 21. Determine. That is, the state of the input / output object 23 with respect to the marker 24 is calculated, and based on this state, conversion parameters of the video to be displayed, for example, the position and inclination of the input / output object 23 and the input / output object 23 are calculated. Conversion parameters for matching the display position of the video to be displayed superimposed, the size of the video, the tilt, and the like are calculated. The conversion parameter calculated in this way is output to the conversion unit 50.

A specific example of this conversion parameter will be described. This conversion parameter is numerical information constituting the conversion matrix M. The transformation matrix M for performing the transformation process is such that the coordinates (X, Y, Z) at which the image is displayed are aligned with the coordinates (x, y, z) of the input / output object 23, and the rotational movement about each axis is performed. It is expressed by a matrix that performs translation (see equation (1)). According to this transformation matrix M, when the input / output object 23 is rotating, the rotation parameter R portion is defined by the rotation angle θa, and in the case of parallel movement, it follows the movement axis of ΔX, ΔY, ΔZ. Parameters are specified. Note that this conversion parameter is not limited to the conversion matrix M, and any conversion parameter can be used as long as it allows the moving / rotating process of the image to be performed. Good.

  The video storage unit 40 is a part that stores an image that matches the input / output object 23. This image may be an image of only input means such as an image for keyboard or pen input, or may be an image in which an input method and an output reflecting this input are combined. Further, only an image drawn on a general display without an input method may be used. Which image to display may be selected in advance by the user, or may be automatically determined according to the content. The video storage unit 40 may be a memory such as a mobile phone provided with the recognition device 100, or may be a server via a communication network. The image stored in the video storage unit 40 is output to the conversion unit 50.

  The conversion unit 50 performs conversion processing such as enlargement / reduction, rotation, and parallel movement on the video stored in the video storage unit 40 based on the conversion parameter output from the conversion parameter calculation unit 30, and displays this on the display unit. 60.

  The display unit 60 displays the converted video. The display unit 60 may be a glasses-type display such as an HMD (Head Mount Display) or a general display. Here, when the display unit 60 is a glasses-type display, the display unit 60 is preferably a transmissive display. By using a transmissive display, an image can be superimposed and displayed on the object viewed through the lens.

  The input detection unit 70 is a part that detects a user input operation performed on the video of the input means displayed on the display unit 60 and receives an input operation. The input detection unit 70 detects an input position of the input / output object 23 on the video, and determines an input made by the user based on the detected input position. Based on the position information detected by the input detection unit 70, gesture recognition and pen recognition can be executed with a virtual keyboard or camera using infrared rays or ultrasonic waves.

  For example, if the input detection unit 70 is a virtual keyboard using infrared rays, the features (size, shape) of the input / output object 23 are transferred from the feature point extraction unit 22 to the input detection unit 70 (infrared sensor (not shown)). ) And the infrared projection range is changed in accordance with the shape of the input / output object 23. Then, the input is determined by measuring the reflectance of infrared rays that changes when the user's finger touches the input / output object 23. In order to realize this processing, the input operation can be detected by storing the position of the finger to input and the position of each keyboard displayed in the projection range in association with each other. Further, when the input detection unit 70 is configured by a camera, the position of the finger and the position of the image displayed on the input / output object 23 are stored in association with each other according to the position of the finger. Processing, for example, an instruction signal is output to the device 80.

  The input detected by the input detection unit 70 is output to the conversion unit 50, the input result is reflected by the conversion unit 50, and displayed on the display unit 60. For example, when a button is pressed, the pressed video is displayed on the display unit 60, and the operated content, for example, a keyboard is displayed. The user presses “K”. In this case, the pressed “K” is displayed.

  The device 80 is, for example, a household electrical appliance such as a television or an information device such as a personal computer, and includes a receiving unit 81 that can receive a signal output from the input detection unit 70. The receiving unit 81 receives the input of the input detection unit 70 and performs processing according to the input. For example, when the input / output object 23 is used as a remote control, for example, the input detection unit 70 receives an input of a remote control operation and outputs the result to the reception unit 81. The receiving unit 81 performs processing based on the result, for example, performs processing for performing channel change processing.

  Here, the relationship between the marker 24 and the input / output object 23 will be described with reference to FIG. In this example, the recognition apparatus 100 adopts a glasses-type shape. The eyeglass-type recognition device 100 includes an object region detection unit 13 that detects an input / output object 23 and a marker detection unit 12 that detects a marker 24 in a frame portion of glasses. The marker 24 can be detected. The display unit 60 is formed inside the lens, and is configured to be able to visually recognize an object through the lens and display an image.

  FIG. 3 is an explanatory diagram showing the relationship between the marker 24 and the input / output object 23. As shown in FIG. 3, the marker 24 is added to the input / output object 23. The input / output object 23 is paper, but is not limited to this and may be a three-dimensional object. The marker detection unit 12 detects the marker 24 based on the information stored in the registration unit 11 as described above. The object area detection unit 13 detects a collection of features synchronized with the movement of the marker 24. Here, the input / output object 23 is detected. The feature point extraction unit 22 can extract feature points (shape, size, etc.) according to the object region detected by the object region detection unit 13. In FIG. 3, since the input / output object 23 is an object formed in a rectangular shape, by extracting two diagonal points, that is, two points that can form the longest line on the object region. The size according to the object area can be recognized. Similarly, four points may be extracted. Moreover, you may make it take a some point, without limiting to rectangles, such as 2 points | pieces and 4 points | pieces.

  The conversion parameter calculation unit 30 calculates a conversion parameter according to the extracted feature points, and the conversion unit 50 processes the video stored in the video storage unit 40 according to the conversion parameter. Here, as shown in FIG. 3, a projection object (for example, paper or a planar object) that is observed through a lens of the glasses on a display (display unit 60) formed inside the user's glasses. Etc.), the keyboard image 25 is displayed in a superimposed manner. The video 25 displays a display portion 25 a and a keyboard portion 25 b, and the user can operate the keyboard in a pseudo manner by touching the paper constituting the input / output object 23. In operation, for example, when a key is pressed, a character corresponding to the pressed position is displayed on the display portion 25a.

  Thus, the user can perform an input operation by operating the keyboard displayed on the display unit 60 while being superimposed on the input / output object 23.

  Next, the operation of the recognition apparatus 100 of this embodiment will be described. FIG. 4 is a flowchart showing the operation of the recognition apparatus 100. First, the marker 24 is attached by the user of the recognition apparatus 100 (S101). And the marker 24 is detected by the marker detection part 12 (S102). Next, the input / output object 23 to which the marker 24 is attached is detected by the object region detection unit 13 (S103). The feature points of the input / output object 23 detected by the object region detection unit 13 are extracted, and the coordinates constituting the input / output object 23 are acquired (S104).

  Next, the conversion parameter calculation unit 30 calculates conversion parameters for converting the tilt, position, size, and the like of the video stored in the video storage unit 40 based on the state of the marker 24 (S105). . Based on the conversion parameters, the video is converted by the conversion unit 50 (S106). The converted video is displayed on the display unit 60 (107). In the present embodiment, since the display is formed on the lens so as to be transmissive inside the glasses, an image is superimposed on the input / output object 23 and displayed.

  The user operates a keyboard that is an image displayed superimposed on the input / output object 23. Then, the operation position on the input / output object 23 of the user is detected by the input detection unit 70, and input processing corresponding to the position is performed (S108). Then, the display image displayed on the display portion 25a is rewritten, for example, the pressed character is displayed (S109).

  Note that since the shape of the input / output object 23 is known at the time of the processing of S107, the operation of S104 to S107 is repeated, so that even if the state of the input / output object 23 changes somewhat, it robustly follows. be able to. When the marker 24 is attached to another object, a switching operation such as turning off the power is required.

  In addition, by performing control so that the operations of S102 to S107 are repeated, the marker detection unit 12 may detect the marker 24 and the object area detection unit 13 may detect the input / output object 23 each time. . In this case, the marker 24 can be attached to another object without an explicit switching operation to recognize the object.

  By omitting the processing of S108 and S109, the object can be used only for the display, and the video can be viewed on the screen of the user's favorite size.

  Further, the input image is not limited to displaying only an input pattern such as a keyboard or character input, and an output image drawn on a general display may be displayed together, or only an output image may be displayed. In addition, a three-dimensional object such as a stuffed animal can be used as an input / output object, and each part can have a function. For example, it is possible to perform a sensory operation such as answering a telephone by pressing a hand.

  Next, operational effects of the recognition device 100 of the present embodiment will be described. In the recognition apparatus 100, the marker detection unit 12 detects a marker 24 that is added to the input / output object 23 that is an identification target and formed so as to be distinguishable from the input / output object 23. The object region detection unit 13 detects the region of the input / output object 23 that is synchronized with the detected movement of the marker, and the feature point extraction unit 22 is based on the region detected by the object region detection unit 13. Feature points such as the size and shape of the input / output object 23 can be extracted. Accordingly, the input / output object 23 can be recognized by simple processing without registering the input / output object 23 in advance.

  Further, in the recognition apparatus 100, the display unit 60 can display an image that matches the feature point with the size and shape of the input / output object 23, and is easy to use such as enlarging or reducing the image to be displayed. A good device can be realized.

  In the recognition apparatus 100, the conversion unit 50 can process the video according to the state of the marker 24 calculated by the marker state calculation unit 21, and the display unit 60 can display the processed video. Therefore, by performing processing such as enlargement, reduction, angle conversion, etc., on the basis of the position, inclination, etc. of the marker 24, it is possible to perform display according to the state of the input / output object 23, and a more convenient device. Can be realized.

  Further, in the recognition apparatus 100, the display unit 60 displays an operation screen according to characteristics such as the size and shape of the input / output object 23, and the input detection unit 70 instructs the input / output object 23 by the user. Detect the operation position. The input detection unit 70 can accept an input operation based on the operation screen and the detected position. Therefore, the input / output object 23 arbitrarily selected by the user can be handled as an input means, and a user-friendly device can be realized. For example, in the case of a small key such as a cellular phone, when the user arbitrarily selects a relatively large object as the input means, an easy-to-use input means can be realized.

It is a block diagram which shows the function structure of the recognition apparatus 100 in this embodiment. 2 is a hardware configuration diagram of a recognition apparatus 100. FIG. It is explanatory drawing which shows the relationship between the marker 24 and the input / output object. 4 is a flowchart showing the operation of the recognition apparatus 100. It is a conceptual diagram which shows the concept of the detection of the input-output object to which the marker using optical flow etc. was added.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Detection part, 11 ... Registration part, 12 ... Marker detection part, 13 ... Object area | region detection part, 20 ... Processing part, 21 ... Marker state calculation part, 22 ... Feature point extraction part, 23 ... Object for input / output, 24 ... Marker, 25 ... Video, 30 ... Conversion parameter calculation unit, 40 ... Video storage unit, 50 ... Conversion unit, 60 ... Display unit, 70 ... Input detection unit, 80 ... Device, 81 ... Reception unit, 100 ... Recognition device, 104 ... sensor, 105 ... display unit, 105 ... auxiliary storage device, 106 ... short-range wireless communication unit.

Claims (5)

Marker detection means for detecting a marker that is added to an object to be identified and formed to be distinguishable from the object;
An object area detecting means for detecting an area synchronized with the movement of the marker detected by the marker detecting means;
A recognition apparatus comprising: feature point extraction means for extracting feature points of the object based on the area detected by the object area detection means. Display means for displaying an image according to the feature point of the object;
The recognition device according to claim 1. State calculating means for calculating the state of the marker;
Conversion means for processing an image displayed on the display means according to the state of the marker calculated by the state calculation means,
The recognition apparatus according to claim 2, wherein the display unit displays the video processed by the conversion unit. Input detection means for detecting a positional relationship between the object and an operation position by the user;
Receiving means for receiving an input operation based on an operation position by the user detected by the input detection means;
With
The said display means displays the image | video which comprises the operation screen according to the characteristic of the said object, The said user can perform operation according to the image | video which comprises an operation screen. The recognition device described in 1. A marker detection step of detecting a marker that is added to an object to be identified and formed to be distinguishable from the object;
An object region detection step for detecting a region synchronized with the movement of the marker detected by the marker detection step;
A recognition method comprising: a feature point extracting step of extracting a feature point of the object based on the region detected by the object region detecting step.

Images