JP2004338889A - Image recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image recognition device capable of analyzing image of a camera loaded on a movable body to perform collection and management of article information automatically. <P>SOLUTION: The image photographed by the movable body provided with the camera 101 is analyzed to constitute the device collecting and managing article information. This device is provided with a movable body control means for controlling movement of the movable body, a movable body parameter inference means for obtaining or inferring information such as position, attitude, and speed of the movable body, an imaging means for obtaining image by controlling position, attitude, and zoom value of the camera, a camera parameter inference means for obtaining or inferring information such as position, attitude, and zoom value of the camera, a symbol recognition means for recognizing characters and symbols giving features to the article from the photographed image, and a position inference means for inferring a position of article. Conventionally, a catalog put in electronic data for management is used to retrieve books in a library, and a book must be confirmed by human labor to confirm matching of information of the catalog with actual situation of the book in management work of books, thereby increasing worker's burden. But, it is possible to solve the problem by this device. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a video recognition device for analyzing a video of a camera mounted on a moving body to collect and manage information on articles.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, for example, libraries and the like use catalogs of books for book management. The catalog is stored as electronic data, and can be easily searched and managed using a computer (for example, see Non-Patent Document 1 below).
[0003]
[Non-patent document 1]
Haruki Nagata, "Present and Future of Library Cataloging: Metadata and OPAC", Information Science and Technology, Vol. 46, no. 3, pp. 106-113, 1996.
[0004]
[Problems to be solved by the invention]
In the management work of books and the like, in addition to the management of electronic data of catalogs according to the above-described conventional technology, it is necessary to manually check the books themselves in order to check the consistency between the information and the actual situation of the books. . This operation places a heavy burden on the operator.
[0005]
Therefore, an object of the present invention is to provide a video recognition device for analyzing a video of a camera mounted on a moving body and automatically executing information collection and management of articles.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, the present invention provides the following means.
(1) Moving body control means for controlling the movement of the moving body, moving body parameter estimating means for acquiring or estimating information such as the position, posture, and speed of the moving body, and controlling the position, posture, and zoom value of the camera. Photographing means for obtaining an image by taking a picture, camera parameter estimating means for acquiring or estimating information such as the position, posture, and zoom value of a camera; and symbol recognition for recognizing characters and symbols characterizing the article from the taken video. Means and an apparatus comprising position estimating means for estimating the position of the article.
(2) Equipped with an information presenting means, the position information of the article on the map of the area where the article is stored, the detailed position information of the article at a place such as a shelf where the article is stored, and the color and shape of the article are displayed. Present an identifiable video.
(3) An information management unit is provided, and the management information of the article is updated using information of the article detected by the symbol recognition unit and the position estimation unit.
(4) A region information estimating unit is provided, and a vacant region in a region for storing articles is estimated using the captured video.
[0007]
BEST MODE FOR CARRYING OUT THE INVENTION
The present invention relates to a video recognition device for analyzing a video of a camera mounted on a moving body and automatically collecting and managing information on articles. Specifically, it can be realized by creating software or hardware for implementing the following method.
[0008]
FIG. 1 is a configuration diagram of the video recognition device in the present embodiment. This device is composed of a group of sensors 101 to 104, a mobile robot 110, a computer 120, a console 130, and an external storage device 140.
[0009]
As the sensors, a camera 101, an acoustic sensor 102, a drive system state measurement sensor 103, and posture measurement sensors 104a and 104b are used. The number of cameras and the number of acoustic sensors are one or more. Parameters such as the focal length and posture of the camera 101 are calibrated in advance, and are used as the initial values of the camera parameters. As the acoustic sensor 102, an active sonar capable of measuring a distance by transmitting a sound wave and observing a reflected wave is used. A rotary encoder or a tachogenerator is used as the drive system state measurement sensor 103, and a gyro or the like is used as the attitude measurement sensor 104.
[0010]
A controller 111 is built in the mobile robot 110. The controller 111 performs acquisition of observation data of the acoustic sensor 102 mounted on the mobile robot, measurement of the state of the driving device 112 (wheel), and control.
[0011]
The computer 120 acquires video data of the camera 101 and performs image processing, and calculates a control amount of the camera 101 and the mobile robot 110. Wireless or wired communication means is used for receiving the video data of the camera 101, receiving the state parameters of the mobile robot 110, and transmitting control commands.
[0012]
The console 130 performs display of camera images, display of the state of the mobile robot 110, display of collected article information, setting of various parameters, and the like.
[0013]
The external storage device 140 stores management data of articles, map data of an area for storing articles, and the like.
[0014]
FIG. 2 is a block diagram of a process in the video recognition device. Hereinafter, each process will be described. In the following, it is assumed that the mobile robot 110 is moved to the vicinity of the bookshelf 301 as shown in FIG. 3, the book 302 on the bookshelf is photographed, and a desired book is searched from the title of the book recognized from the video. I do.
[0015]
The video input unit 211 acquires video data of the camera 101. Preprocessing such as A / D conversion is performed on the data, and the processed video data is output.
[0016]
FIG. 4 shows an example of video data obtained by the video input unit 211. Even when the same book is photographed, different video data can be obtained depending on the position and orientation of the camera 101 and the mobile robot 110.
[0017]
The posture measurement unit 212 measures the postures of the camera 101 and the moving body 110 using the posture measurement sensors 104a and 104b.
[0018]
The drive system state measurement unit 213 acquires the state of the drive device 112, that is, the number of rotations and the rotation angle. From these data, the position, speed, and angle (traveling direction) of the mobile robot 110 are calculated. Here, the position and the angle are obtained as values based on the initial position of the mobile robot 110.
[0019]
The moving body parameter estimating unit 222 calculates the state parameter of the moving body 110 based on the change amount of the state parameter (the position, posture, and speed of the moving body 110) calculated one time before by the moving body control unit 223 described later. Update. Further, the state parameter is corrected using the position and posture data of the moving body 110 obtained by the posture measurement unit 212 and the drive system state measurement unit 221.
[0020]
In the camera parameter estimating unit 213, the amount of change in the camera parameters (the position, posture, and focal length (zoom value) of the camera 101) calculated one time before by the camera control unit 214 described later, and the moving object parameter estimating unit 222. The camera parameters are updated based on the calculated state parameters of the mobile robot 110. Also, the camera parameters are corrected using the posture data measured by the posture measurement unit 212 and the focal length, position, and posture data estimated using a plurality of time-series video data. As a method of estimating parameters using a plurality of video data, a method described in the following document 2 or the like can be used.
[0021]
Reference 2: Susumu Hattori, "CCD Camera Calibration without Reference Point by Multiple Imaging of Target Field", IEICE D-II, Vol. 82, no. 9, pp. 1391-1400, 1999.
The camera controller 214 changes the camera parameters based on the video data acquired by the video photographing unit 211 and the camera parameters calculated by the camera parameter estimator 213 so that the camera 101 can obtain a desired video. Amount) and the control command is determined. Here, the desired image means, for example, an image in which the book 302 on the bookshelf 301 is in focus. The focal length and attitude of the camera are controlled based on the calculated control amount and control command.
[0022]
The mobile control unit 223 calculates a change amount (control amount) and a control command based on the state parameter of the mobile robot 110 calculated by the mobile parameter estimation unit 222 so that the mobile robot 110 takes a desired action. Make a decision. Here, the desired behavior is set in advance such as (a) approaching the bookshelf 301 until the set distance is reached, (b) moving parallel to the bookshelf 301 after approaching, and the like. Based on the calculated control amount and the control command, the number of rotations and the rotation angle of the driving device are calculated, and the driving device is controlled.
[0023]
The symbol recognition unit 231 recognizes a symbol or a character that characterizes the article from the video data. The processing here is roughly classified into three: image correction processing, symbol area extraction processing, and symbol recognition processing. In the image correction process, the image data is geometrically transformed to correct the rotation and distortion of the surface of the article on which symbols, characters, and the like exist. In this processing, video data captured under different conditions is converted into video data captured under certain conditions (for example, captured in a state of facing a book). This processing is executed as needed. In the symbol area extraction processing, an area where a symbol or a character string to be recognized exists is extracted from the video data. When there are a plurality of symbols and characters, each symbol and character are also cut out. In the symbol recognition process, the characteristics of the extracted symbols and characters are analyzed to recognize the contents represented by the symbols and characters. At the time of recognition, the data is compared with the data in the external storage device 140. As these specific methods, various methods described in the following Reference 3 and the like can be used.
[0024]
Reference 3: "IEICE Technical Report (PRMU 96-180 to 191)", IEICE, 1997.
FIG. 5 shows an example in which the image correction processing is performed on the image 401. In this example, the rotation of the title of the book is corrected by this processing, and the image is converted into a video taken from the front of the book. FIG. 6 shows an example in which a symbol area extraction process has been performed on the image 501. As shown in this example, only the title region 602 of the book is extracted by this processing. In addition, each character is cut out at the same time. Each title is recognized by a symbol recognition process using the extracted data.
[0025]
The position estimating unit 232 calculates the position of the article in the real space using the position on the camera image of the article extracted by the symbol recognition unit 231 and the camera parameters estimated by the camera parameter estimating unit 213. Specifically, using a plurality of videos of the article, the intersection of a straight line connecting the camera position at which the video was captured and the detected position of the article on the camera video is calculated. This method is described in the following document 4 including the case where the detection position on each image has an estimation error.
[0026]
Reference 4: Mikio Takagi and Hirohisa Shimoda, "Image Analysis Handbook", University of Tokyo Press, 1991.
The position of the article is obtained as a three-dimensional coordinate value. By storing the correspondence between the coordinates and the position of the bookshelf in the external storage device 140 in advance, it is possible to obtain information as to which book on which bookshelf, at which position there is a book.
[0027]
The information management unit 233 updates the management information of the article in the external storage device 140 using the information of the article extracted by the symbol recognition unit 231 and the position information of the article estimated by the position estimation unit 232.
[0028]
The information display unit 234 presents position information of an item on a map, more detailed position information at a location such as a shelf where the item is stored, and an image that can identify the color and shape of the item.
[0029]
FIG. 7 shows an example in which the result of estimating the position of a book is displayed on the console 130. In FIG. 702, each bookcase 703, the position 704 of the mobile robot 110, and the book position 705 are displayed, and the bookcase where the book is placed can be confirmed on the map. In FIG. 706, each bookcase 707 and book position 708 are displayed, and the position of the book on the bookcase can be confirmed.
[0030]
The symbol recognition unit 231 recognizes a symbol or a character that characterizes an article, and detects the article itself. On the other hand, it is also possible to estimate the free space in the area for storing articles by using a video image of a wide area. As a specific method, an empty area is determined based on the detection result (the number and position of the articles) of the articles by the symbol recognition unit 231 and the position estimation unit 232 and the information (such as the size) of the articles stored in the external storage device 140. presume. In addition, a method of classifying objects in a video described in Document 4 or the like can be used.
[0031]
FIG. 8 shows an example of a video image of a wide area. By estimating the bookcase and book areas 802 and 803 from such an image, the free areas 804 and 805 in the bookcase 301 are estimated.
[0032]
As described above, by automatically moving the mobile robot to capture an image, and by recognizing the symbols and characters characterizing the article from the image, information collection and management of the article can be automatically executed.
[0033]
【The invention's effect】
ADVANTAGE OF THE INVENTION According to this invention, a camera is mounted on a moving body, an image is taken while automatically controlling its movement, and symbols and characters characterizing the article are recognized using the image, thereby collecting information on the article. And a video recognition device that automatically executes management.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a video recognition device.
FIG. 2 is a block diagram of processing in the video recognition device.
FIG. 3 is an example of a shooting situation by a mobile robot.
FIG. 4 is an example of video data obtained by a video photographing unit.
FIG. 5 is an example of video data after a video correction process.
FIG. 6 is an example of video data after a symbol area extraction process.
FIG. 7 is a display example of an article position estimation result.
FIG. 8 is an example of a wide area image.
[Explanation of symbols]
Reference Signs List 101: camera, 102: acoustic sensor, 103: drive system state measurement sensor, 104a, 104b: posture measurement sensor, 110: mobile robot, 111: controller, 112: drive device, 120: computer, 130: console, 140 .., External storage device, 301, bookshelf, 302, book, 401, camera video, 501, video after video correction processing, 601, detection result of characters in the video, 701, information presentation example on the console, 801, wide area video Example.

Claims (4)

It is a device that collects and manages information on articles by analyzing the image of the camera mounted on the mobile robot,
Moving body control means for controlling the movement of the already moving body,
A moving body parameter estimating means for acquiring or estimating information such as the position, posture, and speed of the already moving body,
A photographing means for controlling the position, posture, and zoom value of the already-camera to acquire an image,
Camera parameter estimating means for acquiring or estimating information such as the position, posture, and zoom value of the already-existing camera;
Recognizing means for recognizing characters and symbols characterizing the existing article from the already taken video;
A video recognizing device comprising: a position estimating unit for estimating a position of an existing article. In claim 1,
Presents position information of the existing article on the map of the area where the existing article is stored, position information of the existing article at a place such as a shelf where the existing article is stored, and an image that can identify the color and shape of the existing article. A video recognition apparatus, comprising: In claim 1,
An image recognition apparatus, comprising: an information management unit that updates management information of an existing article using information of the existing article detected by the already recognized unit and the existing position estimation unit. In claim 1,
A video recognition apparatus comprising: area information estimating means for estimating the size of an unused area in an area for storing an existing article by using an already-taken video.

Images