KR100853185B1 - Object recognition and localization method using both vision information and wireless sensor network and apparatus using the same - Google Patents

Abstract

Object recognition and positioning method using the image information and the wireless sensor network according to the present invention and the device is a device for recognizing and measuring the location based on the data provided by the object recognizes the object using the image information and wireless sensor network A method of positioning, the method comprising: obtaining an ID of an object for which a position is to be measured and acquiring information of the object based on the ID; Estimating the location of the thing using wireless communication with the electronic tag attached to the thing; Acquiring an image near the estimated position of the object; And repositioning the location of the object from the acquired image. Since the ID and the accurate location information of various objects existing in the space can be grasped, an intelligent robot such as an object manipulation and delivery can be identified. It can be applied to service development.

Mobile robot, Wireless sensor network, Robot vision Object recognition, Object localization, Sensor fusion

Description

Object recognition and localization method using both vision information and wireless sensor network and apparatus using the same}

1 is a block diagram showing the configuration of an object recognition and positioning apparatus using image information and a wireless sensor network according to the present invention.

2 is a diagram illustrating a method of recognizing and positioning an object using image information and a wireless sensor network and a system to which the apparatus is applied according to the present invention.

3 is a flowchart illustrating a process of object recognition and positioning using image information and a wireless sensor network according to the present invention.

4 is a diagram illustrating an error range of an estimated object position and a visible range of a robot camera.

The present invention relates to a method and apparatus for recognizing objects existing in a service environment and determining the location of each object, and more particularly, to obtain an ID of the object through wireless communication with the object to be located. After requesting and receiving information from a database that stores the information of the thing, and after receiving the image of the object from the device including the robot and the information obtained by measuring the accurate recognition and its location to It is about.

In order for robots to provide useful services to humans, it is necessary to be aware of the environment of the service space, and in particular, it is important to recognize objects in the space and to grasp the location information. For example, in order for a robot to deliver a drink to a human being, the robot must first recognize the presence of a drink and then grasp the location information to access the drink.

The traditional method for recognizing and locating objects is image recognition technology using a camera. In general, image recognition technology uses a method of constructing and storing an image DB for each object in advance and comparing the object with a newly input image. If the object is successfully recognized and detected in the image, the location information of the object may be calculated relatively accurately by using a stereo matching method using two cameras. This image recognition method has been successfully applied in a well structured and fixed environment such as a factory. However, as the application environment of robots is changed from industrial robots to unstructured and complex general environments such as homes, public institutions, and offices, there is a limit to environment and object recognition using only image recognition technology. As the number and type of objects to be recognized increases, the number of DB images to be compared increases, so that the computation time increases according to the comparison and the probability of false recognition or detection also increases. In addition, since the field of view of the camera is limited, it is very inefficient to find objects while the robot moves one by one when the service space where the robot is active is large.

Another method for recognizing and locating objects is wireless communication and sensor network technology, which is being actively researched recently. The RFID system is a system that can recognize the location with low facility investment and low management cost. The RFID system is composed of a tag, a reader, and a data processor. The tag's unique identity is transmitted around by radio waves, and the reader receives the signal and finally interprets the object in the data processing system. The disadvantage of RFID is that the strength of the signal from the tag changes slowly over time and it is difficult to calculate the propagation time of the radio waves. Therefore, it is difficult to determine the location using only RFID.

Positioning methods based on the wireless sensor network include a method using an ultra wideband (UWB) signal and a method using a ZigBee (IEEE 802.15.3a) and a WLAN (IEEE 802.11). The UWB system extracts the first arriving pulse using an extremely short pulse, and then estimates the position of the object by triangulation method using the arrival time difference between the receivers. The advantage of this method is that the position can be recognized at a relatively long distance, and the position of the object can be determined roughly due to the strong propagation power of the UWB signal even in a space blocked by walls or partitions, and compared with other radio waves. It is less affected by pathing, scattering, or diffraction.

On the other hand, the disadvantage of UWB is that the receiver to receive the signal has to be attached to a fixed location on the wall, which requires a lot of equipment for installation, and it is difficult to locate the subdivided area. ZigBee mainly uses the strength of the signal reached, and ZigBee transmitter / receiver is required. In general, ZigBee is known to guarantee position accuracy of about 3 to 6 meters. WLANs also offer position accuracy of about 3 to 6 meters and require an access point suitable for WLAN propagation environments. The advantage of WLAN is that installation cost is low because it uses the existing wireless internet network. However, in case of WLAN, the position of object is determined by comparing the pattern and intensity of radio waves. This requires a lot of installation time and manual and repetitive manual work. In the case of the propagation map, it is easy to be affected by changes in the environment, which makes it impossible to precisely determine the location.

In summary, as described above, it is very important for intelligent services to recognize objects in the service space of the robot and to identify the exact location. However, it is time consuming to find objects using only image recognition in a large space. As a processing technique, there is a high probability of false recognition. On the other hand, the wireless communication network technology can easily recognize the ID of the thing, but there is a problem that can not provide accurate location information due to the propagation characteristics and can provide only approximate location information of about 2 ~ 3m.

The technical problem to be solved by the present invention is to solve the above problems, by combining wireless sensor network technology with image recognition technology to remove the uncertainty due to image processing and reduce the amount of calculation of objects present in the space efficiently The present invention provides a method and an apparatus capable of accurately identifying location information.

In order to achieve the above technical problem, the object recognition and positioning method using the image information and the wireless sensor network according to the present invention uses an image information and a wireless sensor network as a device for recognizing and measuring a location based on data provided by the object. A method for recognizing and positioning a thing, the method comprising: obtaining an ID of an object for measuring a location and obtaining information of the object based on the ID; Estimating the location of the thing using wireless communication with the electronic tag attached to the thing; Acquiring an image near the estimated position of the object; And repositioning the position of the object from the acquired image.

In order to achieve the above technical problem, the object recognition and positioning apparatus using the image information and the wireless sensor network according to the present invention includes a database that stores information of a thing and provides it on request; And recognizing the thing, acquiring the information of the thing provided by the database, estimating the location of the thing through wireless communication, and acquiring an image near the estimated location of the thing to obtain the information and the image of the thing. And a wireless location recognizing unit to redefine the location of the object based on the location of the object.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. 1 is a block diagram showing the configuration of an object recognition and positioning apparatus using image information and a wireless sensor network according to the present invention. 2 is a diagram illustrating a method of recognizing and positioning an object using image information and a wireless sensor network and a system to which the apparatus is applied according to the present invention. Meanwhile, FIG. 3 is a flowchart illustrating a process of a method for recognizing and locating an object using image information and a wireless sensor network according to the present invention, and FIG. 4 is a view illustrating an error range of an object estimation position and a visible range in a robot camera.

First, the configuration of the object recognition and positioning device using the image information and the wireless sensor network according to the present invention will be described. The wireless location recognition unit 110 may be included in the robot 130 or may be configured as a separate device to operate in conjunction with the robot 130. The wireless location recognition unit 110 performs a function of measuring the location of the thing. That is, after detecting the object from the camera image provided by the robot 130 using information data about the object previously stored in the database 120 using the ID of the object, the image provided by the robot 130 and Comparing to calculate the exact position, for this purpose is composed of the information collecting unit 111, the image collecting unit 113, the position calculating unit 115. The information collecting unit 111 obtains the ID of the thing to measure the position and then obtains the information of the thing provided by the database 120 based on the ID. The image collecting unit 113 receives an image from the robot, and the position calculating unit 115 estimates the position of the object and calculates the position of the object based on the received image.

Meanwhile, the database 120 stores the information of the thing, and provides the information when requested by the wireless location recognition unit 110. Meanwhile, the robot 130 obtains an image of an object and provides the image to the wireless location recognition unit 110. Meanwhile, the information collecting unit 111 obtains the ID of the thing by using wireless communication with the electronic tag attached to the thing.

An example in which the configuration of the present invention as shown in FIG. 1 is applied will be described with reference to FIG. 2. The position recognition system of the present invention detects an ID and an approximate location of an electronic tag through wireless communication with a camera attached to a mobile robot 230 connected to a network, an electronic tag attached to an object 240, and an electronic tag. Using a localization network (210, localization network) that can be estimated, a database system (220, which stores information about each object), and detecting and locating objects from an image acquired by a camera using information about the objects It is composed of a calculation module for calculating information (110 of FIG. 1), which may be mounted inside the robot 130 or may be configured as a separate module (apparatus) as described above, as follows. The electronic tag attached to the thing includes a unique identifier ID and has a function of broadcasting its ID to the outside in response to a specific signal from the outside.

The wireless location recognition network has a function of identifying an ID of an electronic tag and calculating a location through wireless communication with the electronic tag. The position recognition method of the electronic tag using the wireless communication uses the UWB (ultrawideband, IEEE 802.15.4a) method of estimating the position by using the arrival time difference of the radio signal and the reception direction of the radio wave, and the reception intensity of the radio signal. Either the ZigBee (IEEE 802.15.3a) method for estimating the location or the WLAN (IEEE 802.11) method using the distribution characteristics of the radio waves at each point may be used, but ideally has excellent permeability against walls and obstacles and multi It is preferable to use the UWB method which is less susceptible to path fading, scattering and diffraction.

In the database system, information about each thing is stored and distinguished by using an identifier ID assigned to an electronic tag as a key index. Information about a thing may be stored in the information on the thing through image processing, and specifically, additional data such as image data, image feature amount, size, shape, etc. registered in advance about the thing may be stored. This thing information can be registered manually in the database, but ideally, the company that manufactures the thing (especially a commercial product) provides information about each thing, and the user uses the identifier ID to access the web. You can access the thing DB through this.

The operation module detects the corresponding object from the camera image by using the DB information about the object. When the object is recognized in a general environment, the shape of the object appears in the image according to the posture of the object (pitch, roll, yaw), the distance from the camera, the positional relationship between the object and the camera direction (viewpoint), and the external lighting relationship. Since the size may vary, a method such as template matching is not applicable. Therefore, in general, a method of extracting feature vectors robust to a viewpoint, a size, an illumination, etc. from an image and comparing the feature vectors of images stored in a DB is used (see Lowe, DG., “Object recognition from local scale-invariant features”). , "In ICCV, pp. 1150-1157, 1999). When object recognition and detection are successful in an image, location information of an object may be relatively accurately calculated using a stereo matching method using two cameras.

Next, the position recognition process of the thing according to the present invention will be described in detail with reference to FIG. 3. The location recognition process is largely obtained by obtaining an ID of the object and information of the object (S310), estimating the position of the object using wireless communication (S320), and obtaining an image of the object from the moving robot (S330). After the recognition of the image, and calculating the position (S340). Take a closer look below.

First, in step S310 of obtaining the ID of the thing and the information of the thing, the electronic present in the space in the wireless location recognition system (including a transceiver installed in the space and capable of wireless communication with the electronic tag and has the configuration of FIG. 1) Broadcast a specific signal in space to detect tags. The electronic tags attached to each object return their ID in response to a specific signal. Then, the wireless location recognition system may analyze the received signal to determine the IDs of objects existing in the space. Next, we use the identified ID to access the database system to get information about things. The imported object information data is used to detect the object in the image recognition step later, and may be used to present a list of objects existing in the space.

In the object position estimating step (S320), the position of the electronic tag, that is, the approximate position information of the object with the electronic tag is calculated using a position recognition method based on wireless communication such as UWB. At present, it is known that UWB can recognize the position with a position accuracy of about 2 ~ 3m. In order to provide data for calculating the exact position, the robot moves to the vicinity of the object using the object position information estimated above. The object position estimation result using wireless communication is inaccurate (2 ~ 3m), so there is a problem that it cannot stop in front of the object. However, since the robot looks and approaches the object, the object is in front of the robot unless the robot passes the object. That is, it is located within ± 90 degrees from the direction of the robot. If the robot stops in front of the estimated position of the object, the range angle at which the object exists is much smaller. The point where the robot stops is the accuracy (or error boundary) of the position estimation by wireless communication, the viewing angle of the camera used (determined by the lens), the maximum distance between the camera and the object that can detect the object through image recognition (the used camera image). It may be determined in various ways in consideration of the resolution of the sensor, the distance between the object and the camera).

As an example, as shown in FIG. 4, the estimated position of the object is p, the error limit of position estimation using wireless communication is e, the maximum distance between the camera and the object that can detect the object in the image is d, and the robot Assuming that the robot moves in a straight line while looking directly at the estimated location of the object, the position at which the robot can recognize the object is between ped and p + ed (but at p on the straight line between the robot and point p). Pk points k away from the robot, and p + k points away k in the opposite direction).

If the camera's field of view is wide, it is recommended that the robot detects the object from the point pe, but in general, since the field of view θ of the camera is limited, as shown in FIG. Attempting to detect a thing can prevent a case from not being detected. Therefore, it is desirable to try to detect objects by moving the robot at regular intervals starting from point R to point p + e-d. If the robot is equipped with a distance sensor that can detect a forward obstacle such as an ultrasonic sensor, ideally, if the obstacle is detected in the middle even before proceeding to the point p + ed, the method can proceed only to the extent that it will not hit the obstacle. have.

Next, in the image recognition step (S330), the object is detected from the image by comparing the object information obtained by using the camera image and the ID of the object. At this time, since the ID of the object is already known, the image recognition problem is replaced by a simple detection problem in the image, thereby minimizing the computation time and the possibility of error due to the image processing. In the conventional general image recognition problem, since it compares the input image and all the object data registered in the database (DB), it takes a lot of time and the possibility of false recognition is relatively high. However, since the ID of the thing can be accurately known from the electronic tag attached to the thing, according to the present invention, there is no need to perform a comparison operation with all the thing data in the DB, and only the data of the thing needs to be compared to the image. The processing reliability can be significantly increased.

Finally, in the precise position calculation step (S340) is calculated by measuring the distance between the camera and the object together with the position information in the image of the object detected in the image recognition step (S330) through a separate device and process. Since the position information calculated using the image information is very precise within a few centimeters, calculating the three-dimensional position of the object is relatively basic compared to the camera, and it can be calculated very easily if only the region of the object is detected in the image. . For example, a triangulation method using stereo matching using two cameras can be used, and these methods can be easily understood by those skilled in the art to which the present invention belongs. The description of the process will be omitted.

Through each of the above steps, the wireless location recognition unit 110 or the mobile robot (when the wireless location recognition unit is mounted) can recognize the IDs of the objects present in the service space to accurately recognize the objects, and also accurately locate the location information. I can figure it out.

The object recognition and positioning method using the image information and the wireless sensor network according to the present invention can also be embodied as computer readable codes on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD_ROM, magnetic tape, floppy disks, and optical data storage, and may also include those implemented in the form of carrier waves (e.g., transmission over the Internet). . The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

As described above, optimal embodiments have been disclosed in the drawings and the specification. Although specific terms have been used herein, they are used only for the purpose of describing the present invention and are not intended to limit the scope of the present invention as defined in the claims or the claims. Therefore, those skilled in the art will understand that various modifications and equivalent other embodiments are possible from this. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

As described above, the object recognition and positioning method using the image information and the wireless sensor network according to the present invention and the effects of the apparatus can be summarized as follows. First, the present invention can be solved by fusing the sensor network and the image technology to recognize the object recognition and position determination that must be complicated by the current image recognition technology. The advantage of this method is that the computation time can be shortened, and current image recognition technology can easily solve the image recognition problem that has a high possibility of error by using ID recognition using electronic tags. In addition, it is possible to precisely measure the position of the object placed in an arbitrary position in a large space.

Second, since the method proposed in the present invention can be used to determine the location of things in a room where a sensor network is installed, it can be used in various fields such as hospitals, schools, and offices. Therefore, the present invention will greatly contribute to improving the intelligence of the robot, and is expected to significantly improve the robot's situational awareness. In addition, since accurate location information of objects can be identified, various intelligent robot services such as delivery service and object arrangement can be created based on this.

Claims (11)

In the method for recognizing and positioning the object using the image information and the wireless sensor network, the device for recognizing and measuring the location based on the data provided by the object, (a) acquiring the ID of the thing to measure the position and then obtaining the information of the thing based on the ID; (b) estimating the location of the thing using wireless communication with the electronic tag attached to the thing; (c) acquiring an image near the estimated position of the object; And (d) repositioning the object by detecting the location information of the object in the acquired image based on the acquired object information; and using the image information and the wireless sensor network Object recognition and positioning method. The method of claim 1, wherein step (a) The object recognition and positioning method using the image information and the wireless sensor network, characterized in that for obtaining the ID using wireless communication with the electronic tag attached to the thing. The method of claim 1, wherein the wireless communication is Object recognition and positioning method using the image information and the wireless sensor network, characterized in that using a wireless communication method including UWB, ZigBee, Wi-Fi. delete The method of claim 1, wherein step (d) Image information and a wireless sensor, characterized in that for acquiring an image with a camera after the device moves to the vicinity of the object using the estimated position of the object, and detecting the position information of the object in the acquired image. Object recognition and positioning method using network. The method of claim 1, wherein step (d) After detecting the position information of the object in the image near the estimated position, the object recognition and positioning method using the image information and the wireless sensor network, characterized in that the position of the object is recalculated using a stereo matching method. . A database that stores information of a thing and provides it according to a request; And After recognizing the thing, the information of the thing provided by the database is obtained, the location of the thing is estimated through wireless communication, and an image near the estimated position is obtained, based on the information of the thing and the image. Wireless location recognition unit for re-defining the location of the thing to the object and object recognition and positioning using a wireless sensor network and image information. The method of claim 7, wherein the wireless location recognition unit The object recognition and positioning apparatus using the image information and the wireless sensor network, characterized in that for calculating the exact position of the object by detecting the position information of the object in the obtained image based on the obtained object information. The method of claim 7, wherein the wireless location recognition unit An information collecting unit which obtains information of the thing based on the ID after obtaining the ID of the thing; A position calculator for estimating the position of the thing using wireless communication with the electronic tag attached to the thing; And And an image receiver configured to acquire an image near the estimated position. And the location calculator re-determines the location of the thing by applying a method including stereo matching based on the acquired image. The method of claim 9, wherein the information collecting unit The object recognition and positioning device using the image information and the wireless sensor network, characterized in that to obtain the ID by using wireless communication with the electronic tag attached to the thing. A computer-readable recording medium having recorded thereon a program capable of executing the method according to any one of claims 1 to 3, 5 and 6.

Images