JP7036874B2 - Code recognition device - Google Patents

Description

Embodiments of the present invention relate to a code recognition device.

It is required to efficiently read barcode information from a plurality of barcodes arranged in a wide range in a space such as a warehouse or a store, for example, a product shelf. Normally, when reading barcode information from barcodes arranged at a plurality of locations, it is necessary to hold the reading unit of the barcode recognition device over each barcode. Therefore, it takes time to read all the barcode information.

On the other hand, there is also a method of photographing an area including a plurality of barcodes and collectively reading the barcode information of the plurality of barcodes from the photographed image. In the case of this method, it is necessary to take a picture from a place away from the barcode and obtain an image in which the barcode can be recognized. Therefore, for example, in the case of a reading unit using a two-dimensional image sensor, the light receiving element has a high resolution and a large size. It had to be an area. However, in a barcode recognition device provided with a reading unit that does not have sufficient resolution, the quality of the barcode in the captured image is poor, and the barcode information may not be recognized. Further, since the image is taken from a position away from the barcode, a high-quality image may not be captured depending on the situation at the time of shooting, and the barcode information may not be recognized from the image.

Japanese Unexamined Patent Publication No. 9-022437

An object to be solved by the present invention is to provide a code recognition device capable of detecting barcodes arranged at a plurality of locations from distant positions and efficiently reading barcode information.

According to the embodiment, the code recognition device includes a reading means, a region detecting means, a position detecting means, a first extraction means, and a second extraction means. The reading means uses a camera equipped with a pan / tilt function to shoot toward a shooting range including a code image to be read. The area detecting means detects the code area and the character area included in the code image from the code image captured by the reading means. The position detecting means detects the position of the code area based on the position of the code area in the code image and the pan / tilt information indicating the direction of the camera adjusted by the pan / tilt function. The first extraction means extracts the first code information represented by the code image from the code area. The second extraction means extracts the second code information represented by the code image from the character area in parallel with the extraction of the first code information by the first extraction means. The area detecting means adjusts the direction of the camera by the reading means at a plurality of different positions by the pan / tilt function, and detects the code area from a plurality of code images captured respectively. The position detecting means of the code area is based on the position of the code area detected from the plurality of code images and the pan / tilt information indicating the orientation of the camera at the time of shooting at the plurality of different positions. Detects the position in the three-dimensional space.

The external view of the bar code recognition apparatus in 1st Embodiment. The block diagram which shows the device configuration of the bar code recognition apparatus in 1st Embodiment. The figure which shows the reading range (photographed area) by the bar code reading part in 1st Embodiment. The figure which schematically showed the detailed structure of the bar code reading part in 1st Embodiment. The figure which showed schematically the method for searching the bar code image corresponding to the bar code included in the photographed image in 1st Embodiment. The flowchart which shows the 1st method of reading a bar code by the bar code recognition apparatus of 1st Embodiment. The flowchart which shows the 2nd method of reading a bar code by the bar code recognition apparatus of 1st Embodiment. The flowchart which shows the 3rd method of reading a bar code by the bar code recognition apparatus of 1st Embodiment. The flowchart which shows the 4th method of reading a bar code by the bar code recognition apparatus of 1st Embodiment. The figure which shows an example of the bar code image in 1st Embodiment. The figure which showed schematically the method for searching the label image corresponding to the label included in the photographed image in 1st Embodiment. The flowchart which shows the 5th method of reading a bar code by the bar code recognition apparatus of 1st Embodiment. The block diagram which shows the device configuration of the bar code recognition apparatus in 2nd Embodiment. The figure which shows the external view of the photographing apparatus for photographing the product shelf which contains the bar code area and the character area in 2nd Embodiment. The figure which showed the positional relationship between the shelf reference plane in which a bar code is arranged in 2nd Embodiment, and the visual field reference plane at the time of shooting obtained from a shooting apparatus and a shooting parameter. The figure which showed the relationship with each parameter shown in FIG. 15 in the field of view when the bar code is arranged on the shelf reference plane in 2nd Embodiment. The figure which showed the relationship shown in FIG. 15 and FIG. 16 more generally. The flowchart which shows the method of reading the barcode by the barcode recognition apparatus of 2nd Embodiment. The flowchart which shows the 1st modification which reads the barcode by the barcode recognition apparatus of 2nd Embodiment. The figure which shows the 2nd modification which reads the barcode by the barcode recognition apparatus of 2nd Embodiment.

Hereinafter, this embodiment will be described with reference to the drawings.
(First Embodiment)
FIG. 1 is an external view of the barcode recognition device 1 according to the first embodiment. The barcode recognition device 1 shown in FIG. 1 is a so-called handheld terminal that can be held and operated by the user. The barcode recognition device 1 is a device that optically detects a code attached to a space such as a warehouse or a store, for example, a product shelf or a product, and reads the code information represented by the code. In the first embodiment, for example, a bar code is targeted as a code, and the bar code information represented by the bar code is read. Generally, the barcode includes a striped pattern (barcode pattern) composed of a plurality of striped lines of different thicknesses and a character string (numeric string) indicating the barcode information represented by the barcode pattern. .. The barcode recognition device 1 may recognize not only a barcode but also various codes represented by a combination of a two-dimensional code or a geometric figure.

The barcode recognition device 1 has a barcode reading unit 2 for photographing a barcode, a display unit 3 for displaying a barcode reading status, operation information, reading results, etc., and an input unit 4 for inputting operation commands and the like. It is equipped with. The operator holds the housing of the barcode recognition device 1 and directs the barcode reading unit 2 to the photographing range (photographed area) including the barcode to be read by operating the input unit 4. (Reading of barcode) can be instructed. The barcode recognition device 1 displays a plurality of products such as product shelves, and reads a plurality of barcodes at once by setting a plurality of barcodes attached to the display positions of the products as a shooting range. be able to.

FIG. 2 is a block diagram showing a device configuration of the barcode recognition device 1 according to the first embodiment. As shown in FIG. 2, the barcode reading unit 2, the display unit 3, and the input unit 4 are connected to each other via the data bus 10. Further, a ROM 5 for recording programs and data, a RAM 7 for temporarily recording information, a CPU 6 for processing programs as a central processing processor, a communication unit 8 for communicating with the outside of the device, and a connection interface with the outside of the device ( The I / F unit 9 that becomes the I / F) is connected to each other via the data bus 10. The program recorded in the ROM 5 includes a barcode recognition program that reads barcode information from an image taken by the barcode reader 2. The CPU 6 has an area detection function for detecting a barcode area and a character area included in a barcode from an image taken by the barcode reader 2 by executing a barcode recognition program, and barcode information from the barcode area. The first extraction function for extracting and the second extraction function for extracting barcode information from the character area are realized.

FIG. 3 is a diagram showing a reading range (photographed area 11) by the barcode reading unit 2 when reading a barcode using the barcode recognition device 1 in the first embodiment. The bar code recognition device 1 executes bar code reading with the shooting direction of the bar code reading unit 2 directed toward the bar code 12 at a position away from the bar code 12 arranged in the space. The imaged area 11 by the barcode reading unit 2 can include a plurality of barcodes 12 arranged in the space. The imaging region boundary 13 representing the range of the imaging region 11 is explicitly indicated by a marker such as a laser emitted from the barcode recognition device 1 so that the user can easily identify the imaging region boundary 13.

FIG. 4 is a diagram schematically showing a detailed configuration of the barcode reading unit 2 in the first embodiment. The barcode reading unit 2 has, for example, a photographing lens 16 for photographing, a two-dimensional image sensor 15, an LED light source for illumination 17, and a laser pointer 19. The lighting LED light source 17 outputs, for example, near-infrared light (IR light) 18 in order to illuminate the area to be imaged 11 with an illuminance necessary and sufficient for photographing. The laser pointer 19 irradiates the visible laser light 20 such as red in the photographing direction by the barcode reading unit 2, and displays the photographing area boundary line 13 so that the user can grasp the range of the imaged area 11.

5 (A) and 5 (B) are methods for searching for a barcode image 12a (code image) corresponding to a barcode 12 included in a captured image 21 captured by the barcode reading unit 2 in the first embodiment. Is a diagram schematically showing. The captured image 21 is captured at a resolution at which the barcode image 12a corresponding to the barcode 12 can be acquired, for example, about 200 dpi or more. The barcode image 12a includes a striped pattern (barcode pattern) composed of a plurality of striped lines having different thicknesses.

As shown in FIG. 5A, the CPU 6 sets a search area 24 having a small area with respect to the captured image 21, and sequentially scans the captured image 21 in the horizontal direction 24H and the vertical direction 24V. Here, the barcode is, for example, an EAN (European Article Number) code or a JAN (Japanese Article Number) code, and a one-dimensional barcode (bar code pattern) and a number string represented by the one-dimensional barcode (decoding information) are used. It shall be written together. It is assumed that the number string contains a checksum for confirming correctness.

By scanning with the search area 24, the CPU 6 has a bar code area 22 corresponding to the one-dimensional bar code (bar code pattern) included in the bar code image 12a and a numerical string as shown in FIG. 5 (B). Each candidate of the corresponding character area 23 is extracted. The detection function (detector) that detects candidates for the barcode area 22 and the character area 23 selects feature points that describe characteristic patterns of barcodes and characters, and uses machine learning technology to create teacher data. It consists of a dictionary and a detector created by learning anti-teacher data. The detection function (detector) for detecting the candidate of the barcode region 22 can be configured to pay attention to the period of the vertical component and the horizontal component of the pattern and detect it on a rule basis without learning. In the scanning by the search area 24, the areas extracted as the candidates of the barcode area 22 or the candidates of the character area 23 are combined while considering the barcode pattern and the direction of the digit string, and the candidates of the barcode area 22 and the character area 23 are combined. As a candidate for, it is extracted together with the position information (coordinate position) in the captured image 21.

Next, the operation of the barcode recognition device 1 in the first embodiment will be described with reference to the flowcharts shown in FIGS. 6, 7, 8, and 9.
FIG. 6 is a flowchart showing a first method of reading a barcode by the barcode recognition device 1 of the first embodiment.

The CPU 6 causes the barcode reading unit 2 to take a picture of the imaged area 11 in response to an operation by the user (ActA1). The CPU 6 acquires the captured image 21 as shown in FIG. 5 by photographing the barcode reading unit 2, and searches for the barcode area 22 (ActA2).

Next, the CPU 6 cuts out the barcode area 22 as a rectangular image by further performing a detailed search on the area extracted as a candidate (bar code candidate area) of the barcode area 22 by a nearest neighbor search or the like (ActA3). .. If the barcode region 22 is geometrically distorted due to the captured image 21 being captured from an angle, the cutout region distortion may be corrected by projective transformation or the like (ActA4). After that, the CPU 6 performs a decoding process on the barcode pattern included in the barcode area 22 (ActA5). Here, if decoding can be performed according to a predetermined barcode convention, the CPU 6 outputs the decoding information (bar code information) together with the barcode position information after confirming the checksum (ActA6, OK) (ActA7).

On the other hand, if decoding fails in the decoding process for the barcode pattern (ActA6, NG), the CPU 6 searches for nearby character area candidates existing in the vicinity of the periphery of the barcode area 22 (ActA8), and the character area 23 Cut out as a candidate (character candidate area) (ActA9). Note that the CPU 6 may perform cutout region distortion correction processing for correcting geometric distortion in the character candidate region, if necessary, as in the case of distortion correction of the barcode region 22 (ActA10).

The CPU 6 performs an OCR process after performing a process of removing the barcode pattern from the character area candidate, and recognizes the character string in the character area candidate (ActA10). That is, an attempt is made to acquire a numeric string representing the barcode information. The CPU 6 performs a checksum on the digit string obtained by the OCR process in the same manner as the decoding of the barcode pattern, and if the barcode convention is satisfied (ActA11, OK), the obtained digit string is used as a barcode. It is output together with the position information as the decoding information (bar code information) of (ActA7). If it is determined that the decoding has failed as a result of performing the checksum (ActA11, NG), the CPU 6 executes read error processing (ActA12).

When a plurality of barcode images 12a are searched for in the captured image 21, the CPU 6 repeatedly executes the above-described processing (ActA2 to A7) for each barcode image 12a. As a result, the barcode recognition device 1 can collectively read barcode information from a plurality of barcodes 12 from the captured image 21 once captured by the barcode reading unit 2.

By performing the above processing, for example, as shown in FIG. 10A, when the barcode pattern in the barcode area 22 is clear, the barcode information is output by the decoding process for the barcode pattern. Can be done. On the other hand, as shown in FIG. 10B, even if the barcode pattern in the barcode area 22 has a resolution and sharpness insufficient for the decoding process, the barcode information is obtained by the OCR process for the character area 23. Can be obtained accurately. In general, the resolution required for OCR of a numeric string may be lower than the resolution required for decoding a barcode. Therefore, even if the barcode 12 is photographed with a resolution or sharpness that cannot be decoded, for example, even if the imaged area 11 is photographed at a position far from the barcode, the correct barcode is used. It becomes possible to acquire information. Alternatively, since the barcode 12 can be read even if the barcode reading unit 2 is not equipped with the image sensor 15 having a high resolution and a large area, the cost of the barcode recognition device 1 can be reduced. ..

In this way, in the first method of the first embodiment, the bar code is read by the OCR process to read the numerical string (character string) representing the information equivalent to the bar code information arranged in the vicinity of the periphery of the bar code area 22. It can be used complementarily when the decoding information cannot be acquired from the area 22. This makes it possible to detect barcodes arranged at a plurality of locations from distant positions and efficiently read barcode information. In the first embodiment, the barcode information can be read in the same manner by the second to fourth methods described below.

FIG. 7 is a flowchart showing a second method of reading a barcode by the barcode recognition device 1 of the first embodiment. The second method will be described as a process different from that of the first method shown in FIG.
In the first method, when the decoding process of the bar code pattern (bar code area 22) fails, the neighborhood character area candidates existing in the vicinity of the periphery of the bar code area 22 are searched. In the second method, when the candidate (bar code candidate area) of the barcode area 22 is cut out (ActB3), the neighborhood character area search existing in the vicinity of the periphery of the barcode area 22 is executed (ActB3a), and the character area candidate is executed. The area that does not include the above is determined as the bar code area 22 and cut out. At that time, the CPU 6 temporarily records the position (coordinates) / size information indicating the detected character candidate area in the RAM 7 (memory buffer).

Similar to the first method, the CPU 6 cuts out a character candidate area using the position / size information recorded in the RAM 7 when the decoding process of the barcode area 22 fails (ActB6, NG). Is executed (ActB8). In the second method, the process of searching around the barcode area 22 is not repeated in each of the time of cutting out the barcode area 22 (ActB3) and the time of cutting out the character area 23 (ActB8), so that it is more efficient. Bar code recognition processing becomes feasible.

FIG. 8 is a flowchart showing a third method of reading a barcode by the barcode recognition device 1 of the first embodiment. The third method will be described as a process different from that of the first method shown in FIG.
In the third method, after the captured image 21 is acquired by the barcode reading unit 2, an image area search process for the entire screen of the captured image 21 is executed (ActC3), and the barcode area 22 and the character area 23 are executed. Search for both of them at once. The CPU 6 records position / size information in the RAM 7 (memory buffer) for each of the barcode candidate area and the character candidate area detected by the image area search process.

When cutting out the barcode area 22, the CPU 6 cuts out the barcode area 22 by using the position / size information of the barcode candidate area recorded in the RAM 7 (ActC3). Further, when cutting out the character area 23, the CPU 6 cuts out the character area 23 by using the position / size information of the character candidate area recorded in the RAM 7 (ActC8).

In the third method, the position / size information is collectively recorded for both the bar code candidate area and the character candidate area detected by the entire in-screen search at one time, so that the number of in-screen search processes is reduced. , More efficient barcode recognition processing can be realized.

FIG. 9 is a flowchart showing a fourth method of reading a barcode by the barcode recognition device 1 of the first embodiment.
In the fourth method, the output processing (ActD3 to D7) of the decoded information (barcode information) from the cutout of the barcode area 22 and the output processing (ActD8 to D12) of the decoded information (barcode information) from the cutout of the character area 23 are performed. ) And are executed in parallel, and each decoding result is integrated and output. The CPU 6 executes the image area search process in the same manner as in the third method, and records the position / size information for each of the barcode candidate area and the character candidate area (ActD2). The CPU 6 processes the barcode area cutout (ActD3) and the character candidate area cutout (ActD8) by using the position / size information for each.

The CPU 6 has bar code decoding information (first barcode information) output by the decoding process for the barcode area 22 and OCR decoding information output by the decoding process (OCR processing) for the character area 23. Decode information integration processing for integrating (second barcode information) based on the position / size information about the barcode area 22 and the character area 23 is executed (ActD13).

In the decoding information integration process, when the decoding information corresponding to each of the barcode area 22 and the character area 23 is obtained, the CPU 6 sets the barcode area 22 and the character area 23 based on the position / size information. Is included in one barcode 12 or not. When it is determined that the barcode is included in one barcode 12, the CPU 6 excludes the barcode information output for either the barcode area 22 or the character area 23. Further, in the decoding information integration process, when the decoding information is obtained only for either the barcode area 22 or the character area 23, the CPU 6 reads the decoding information of the barcode 12 indicated by the position / size information. As a result. The CPU 6 repeatedly executes the above-described processing for each of the plurality of barcodes 12 detected from the imaged area 11.

In the fourth method, the barcode area 22 and the character area 23 are decoded in parallel, and the results are integrated. Therefore, even if the barcode area 22 cannot be detected for some reason, the character area 22 is detected. By the OCR processing for 23, it becomes possible to obtain the decoding information (bar code information) represented by the bar code 12 in a complementary manner.

Next, an example in which a code image different from the bar code image 12a described above is processed will be described. Here, a label attached as a management tag for managing a shelf on which a product or an article is displayed, such as a price tag of a product, will be described. Labels are printed, for example, in a predetermined format and posted on goods or display shelves.

11 (A) and 11 (B) are diagrams schematically showing a method for searching for a label image 25a corresponding to a label included in a captured image 21 captured by the barcode reading unit 2 in the first embodiment. Is. In the bar code image 12a shown in FIG. 5 (B), the character area 23 is provided near the periphery of the bar code area 22, but in the label image 25a shown in FIG. 11 (B), the character area 23 is separated from the bar code area 22A. A character area 23A is provided at the position. The label image 25a includes label information 26 in addition to the barcode area 22A and the character area 23A.

As shown in FIG. 11A, the CPU 6 sets a search area 24 having a small area with respect to the captured image 21 in the same manner as described above, and sequentially scans the captured image 21 in the horizontal direction 24H and the vertical direction 24V. Then, the label area of the format shown in FIG. 11B is detected.

FIG. 12 is a flowchart showing a fifth method of reading a barcode by the barcode recognition device 1 of the first embodiment. In the fifth method, as shown in FIG. 11B, a process of reading the barcode area 22A and the character area 23A provided apart from each other in the label image 25a is shown. The fifth method will be described as a process different from the first method shown in FIG.

In the label image 25a, although the character area 23A is not arranged close to the barcode area 22A, it is utilized that the character area 23A is arranged at a predetermined position in the label image 25a. That is, when the arrangement relationship between the candidate of the barcode area 22A and the candidate of the character area 23A corresponds to the format shown in FIG. 11B, the CPU 6 detects it as a label area corresponding to the label image 25a (ActE2). The CPU 6 temporarily records the position / size information indicating the detected label area in the RAM 7 (memory buffer) as the label area information (ActE3). The CPU 6 repeats the process of cutting out the label area for each of the plurality of label images 25a included in the captured image 21 (ActE2, E3).

Next, the CPU 6 searches the barcode area 22A from the label image 25a based on the detected label area information, and decodes the barcode (bar code pattern) of the barcode area 22A (ActE4 to E7). If the barcode decoding result is checksum NG (ActE8, NG), the CPU 6 uses the position / size information recorded as the label area information recorded in the RAM 7 when cutting out the character area 23A. The character area 23A is searched for the label image 25a (ActE10), and the character area 23A is cut out (ActE11).

In this way, when searching for the character area 23A, it is possible to efficiently search for the character area 23A by utilizing the label area information. By performing the preprocessing for detecting the predetermined label image 25a (label area) from the captured image 21 in this way, even if the barcode area 22A and the character area 23A are separated from each other, it is efficient and highly accurate. , The barcode area 22A and the character area 23A can be searched. Further, it is possible to determine whether or not the candidate of the barcode area 22 and the candidate of the character area 23 are in the label, and it is possible to improve the detection accuracy especially when an unrelated barcode is photographed together.

(Second Embodiment)
The barcode recognition device 1A of the second embodiment reads barcode information for the barcode area 22 and the character area 23 in the same manner as in the first embodiment. Further, in the barcode recognition device 1A, instead of the barcode reading unit 2 in the first embodiment, the image pickup device 2A having a fixed installation location is used as a reading device for capturing an image including a barcode image or the like. Detects the position (three-dimensional position coordinates) of the barcode 12 in the three-dimensional space. FIG. 14 shows an external view of a photographing device 2A for photographing a product shelf or the like including a barcode area 22 and a character area 23 in the second embodiment.

The photographing device 2A has a pan (P) function (pattern unit 2A1) for rotating the camera in the horizontal direction and a tilt (T) function (tilt unit 2A2) for rotating the camera in the vertical direction, so-called pan / tilt camera (PT). Camera). The center of rotation of pan / tilt is common, and even if the camera direction is set to an arbitrary rotation angle (θ, φ), the viewpoint position of the camera is fixed at one point. The pan / tilt function is often used in surveillance cameras such as security cameras fixedly mounted on the ceiling or the like, and can be similarly applied to the photographing device 2A. It is also possible to use a photographing device having a zoom (Z) function in addition to the pan / tilt function, that is, a pan / tilt / zoom camera (PTZ camera).

FIG. 13 is a block diagram showing a device configuration of the barcode recognition device 1A according to the second embodiment. As shown in FIG. 13, the photographing apparatus 2A corresponds to the barcode reading unit 2 shown in FIG. Further, the barcode recognition device 1A can be provided with a plurality of photographing devices 2A and 2B. In the barcode recognition device 1A, the system configuration corresponding to the device function 1B such as the CPU, RAM, and communication unit shown in FIG. 2 can be constructed on a locally connected personal computer (PC), an on-premises server, or the cloud. It is possible. The details of the device function 1B are almost the same as those of the first embodiment, and the description thereof will be omitted. The device function 1B inputs images taken by the photographing devices 2A and 2B and executes the process described below.

15 (A) and 15 (B) show a reference surface to be photographed, for example, the label display surface (shelf reference surface 31) of the product shelf is subject to the photographing conditions of the photographing apparatus 2A (PT (PTZ) camera). It is a figure which showed what kind of relationship is geometrically. In FIG. 15, the x-axis and y-axis of the orthogonal coordinate system are set on the shelf reference plane 31, the photographing device 2A is on the z-axis for simplicity, and the tilt angle θ is set to 0.

The viewpoint 28, which is the installation reference position of the photographing device 2A, is arranged at the z-axis coordinate from the origin of the orthogonal coordinate system, that is, at the position of the distance shelf-camera reference distance (zs) from the shelf, and the angle φ in the pan direction is φ0. I made it. Assuming that the shooting horizontal angle of view of the shooting device 2A is 2ω0 and the distance from the camera axis to the shelf, that is, the distance between the shelf and the camera is z0, the shooting horizontal angle of view boundary of the shooting device 2A and the optical axis and the shelf reference surface 31 The intersection positions and distances are as shown in FIG. 15 (B). Further, the shelf shape projected on the visual field reference surface 32 (width 2W) is distorted into a trapezoidal shape like the shelf-visual field reference projection surface 33 shown in FIG. 15A, assuming that the shelf is a rectangular parallelepiped. The right side YR and the left side YL of the shelf-field reference projection surface 33 shown in FIG. 15A are represented by the following equations, respectively.

16 (A) and 16 (B) are views showing the relationship with each parameter shown in FIG. 15 in the visual field range 34 when the barcode 12 is arranged on the shelf reference surface 31. If the reference distance (zs) between the shelf and the camera, the pan direction angle φ0, and the shooting horizontal angle of view ω0 are known from FIG. 16, these parameters and the distance from the optical axis in the field-of-view reference plane 32 to the bar code 12 are obtained. The relationship with w is shown, and it can be seen that the coordinates on the x-axis of the shelf reference plane 31 can be restored. That is, it is possible to calculate the position information of the barcode 12 in the three-dimensional space on the actual shelf from the position information of the barcode 12 in the image captured by the photographing device 2A.

FIG. 17 is a more general representation of the relationships shown in FIGS. 15 and 16. The photographing device 2A (viewpoint 28) exists at (0,0, −zs) in the Cartesian coordinates (x, y, z) existing on the shelf reference plane 31, and is located on the local coordinate system XY two-dimensional plane of the viewing reference plane 32. The reference coordinates and the position of the bar code 12 are represented by a unit spherical coordinate system (θ, φ) on the y'axis in which the y-axis is moved in parallel in the z-axis direction. For example, in the example shown in FIG. 17, the position of the barcode 12 is (θ0, φ0 + ω), and the shelf reference plane 31 is finally reached from the position on the local coordinate system XY orthogonal axis of the visual field reference plane 32 described above. It is possible to project onto the coordinate system of. The reference distance (zs) between the shelf and the camera may be fixedly given as a preset value, or an optical measuring means such as a laser range finder may be provided. It is also possible to indirectly estimate the distance from the lens conditions and the focus position of the photographing apparatus 2A (camera). Similarly, since the distance from the photographing device 2A (camera) to the barcode 12 can be directly or indirectly measured by the method described above, the position information of the barcode 12 can be measured two-dimensionally on the shelf reference plane. It is possible not only to map on the coordinates but also to map on the three-dimensional coordinates.

FIG. 18 is a flowchart showing a method of reading a barcode by the barcode recognition device 1A of the second embodiment. In the second embodiment, the vicinity area is photographed based on the processing (ActF1 to F4) (coarse search processing S1) in the previous stage of the rough search for the barcode candidate area and the barcode position information detected by the rough search processing S1. The shooting is performed in two stages of the subsequent processing (ActF6 to F9) (detailed search processing S2).

In the rough search process S1, the CPU 6 sets the shooting direction (camera orientation condition) of the shooting device 2A and executes shooting (ActF2, F3), and searches the shot image for the barcode image (barcode area 22). Is executed (ActF4). Here, instead of performing an accurate search, a rough search for searching for a candidate for a barcode area with a light processing load is executed, and a barcode candidate position (position / size information) indicating a location to be a candidate for the barcode area is set in the RAM 7. Record in (ActF5). The CPU 6 shoots while changing the shooting direction of the shooting device 2A, searches from each shot image, and records the shooting barcode candidate positions for the candidates of the barcode region (Act F2 to F4).

Next, in the detailed search process S2, the CPU 6 sets the shooting direction (camera orientation) of the photographing device 2A according to the camera orientation condition setting in the rough search process S1 and executes shooting (Act F6, F7). The CPU 6 uses the shooting bar code candidate positions corresponding to the candidates of each bar code area to read the bar code information for the bar code area 22 and the character area 23 in the same manner as in the first embodiment. (ActF9). It should be noted that the processing of ActF9 will not be described as the processing of ActA2 to A11 in the first embodiment shown in the broken line of FIG.

In this way, in the second embodiment, the detailed search process S2 for the barcode area 22 and the character area 23 is executed for the barcode candidate position searched by the rough search process S1, so that the shooting and search time can be increased. It is possible to shorten the processing time by improving efficiency.

Further, in the second embodiment, it is also possible to use a pan / tilt / zoom camera (PTZ camera) having a zoom (Z) function as the photographing device 2A. In this case, in the rough search process S1, the camera zoom initial setting is executed for the photographing device 2A (ActF1) so that the bar code image of the quality required for the rough search can be obtained, and the photographing is executed. Further, in the detailed search process S2, the CPU 6 sets the camera zoom so that a bar code image finer than that in the rough search process S1 can be obtained, and executes shooting (ActF7). That is, by using the zoom (Z) shooting function, the barcode information is recognized for the high-definition barcode image in the detailed search process S2, so that the recognition accuracy can be expected to be improved.

FIG. 19 is a flowchart showing a first modification of reading a barcode by the barcode recognition device 1A of the second embodiment. The process shown in FIG. 19 is based on the process shown in FIG. 18, and further includes the product shelf master information (planogram) regarding the product shelf to which the barcode is attached and the position of the barcode read from the actually captured image. A process is provided to output the result of comparison and reference with the information as a report. The product shelf master information (planogram) is, for example, a database of a product shelf allocation plan, and is information indicating which product is stored in which position on the product shelf.

Here, for example, on the premise that the barcode information posted on the product shelves does not change significantly, it is assumed that the shelf information collation processing is periodically performed a plurality of times. When the CPU 6 reads the barcode information of the barcode attached to the product shelf for the first time, or when the previous reading result is reset and read again (ActG1, Yes), the rough search process S1 shown in FIG. 18 The barcode candidate position is detected (ActG2) and recorded in the RAM 7 as barcode candidate position information (H1) (ActG3).

When executing the second and subsequent readings (ActG1, No), the CPU 6 refers to the barcode position information (H1) and executes the information acquisition process by the detailed search process S2 shown in FIG. 18 (ActG4). In ActG4, the CPU 6 can start shooting the barcode candidate area in the subsequent stage and detailed search processing S2 (ActG4) without executing the entire shooting of the target area and the rough search processing S1 (ActG2).

When more appropriate barcode position information is obtained in the shooting of the barcode candidate area and the information acquisition process (ActG4), the CPU 6 updates the barcode position information by writing the barcode position information in the barcode candidate position information (H1). Perform processing. On the other hand, the CPU 6 records in the RAM 7 the barcode position / decoding information for the barcode candidate area obtained as a result of shooting the barcode candidate area and the information acquisition process (ActG4).

The CPU 6 executes a shelf information collation process for comparing and collating the barcode position / decoding information with the product shelf master information (planogram) (H2) (ActG5). In the shelf information collation process, the CPU 6 determines whether the barcode position information and the barcode information read from the captured image match the product shelf master information. If it is determined that they match, the CPU 6 confirms that the actual shelving allocation of the product shelves is performed correctly and as planned, displays a warning about the mismatched parts, and outputs the results together as a report (H3). do.

By detecting the barcode position information (barcode position / decode information) in this way, it is possible to compare with the product shelf master information, and the correspondence between the position information of goods and shelves and the barcode information can be achieved. The necessary inventory work and inspection work can be facilitated. Since the photographing device 2A can take omnidirectional photography from the place where the photographing device 2A is installed, it is possible to automatically analyze the product shelves around the photographing device 2A, and it is possible to save labor and labor.

FIG. 20 is a diagram showing a second modification of reading a barcode by the barcode recognition device 1A of the second embodiment. In FIG. 20, a plurality of (two in FIG. 20) photographing devices 2A and 2B are used to photograph a common range. In FIG. 20, the distance between the photographing device 2A and the installation position of the photographing device 2B is R. The bar code recognition device 1A searches for and decodes the bar code position based on the above-mentioned processing for the image taken by the viewpoint 28 (shooting device 2A) which is the first camera shooting position, and then the first At the viewpoint 29 (shooting device 2B), which is the camera shooting position of 2, the barcode position is similarly searched and decoded.

When such processing is performed, a certain bar code 12 becomes the position coordinates (r1, θ1, φ1) in the local coordinate system (x1, y1, z1) based on the first viewpoint 28 and the second viewpoint 29. Of the position coordinates (r2, θ2, φ2) in the local coordinate system (x2, y2, z2) based on, it is possible to obtain at least the values of the components (θ1, φ1) and (θ2, φ2) on the unit spherical coordinates. Is. Here, if the distance R and the direction between the first viewpoint 28 and the second viewpoint 29 are known, the coordinates (xc, yc, zc) of the second viewpoint 29 with respect to the first viewpoint 28 can be known. From the values of the position coordinates (θ1, φ1) and (θ2, φ2) of the bar code obtained from the local coordinates at the two viewpoints, for example, an unknown distance component r1 using a set of position coordinate components in a plurality of bar codes 12. , R2, and finally the three-dimensional position coordinates of the bar code in the global coordinate system can be obtained.

Here, the relative position coordinates (xc, yc, zc) of the first viewpoint 28 and the second viewpoint 29 are known, and the distance components r1 and r2 of the barcode 12 in each local coordinate are unknown. However, since the distance r from the photographing devices 2A and 2B (camera) to the barcode 12 can be known by the photographing parameters of the camera or the distance measuring means such as laser ranging, a plurality of barcodes can be obtained. From the set of the local position coordinates (r1, θ1, φ1) and (r2, θ2, φ2) of 12, the position coordinates (xc, yc, zc) of the second viewpoint 29 with respect to the first viewpoint 28 can be obtained. That is, even if the position of the second viewpoint 29 is unknown, it is possible to obtain the three-dimensional position coordinates of each barcode 12. Further, even if the distance components r1 and r2 of the bar code 12 and the position coordinates (xc, yc, zc) are unknown, the positions (θ1, φ1) and (θ2, φ2) between the two points for a large number of bar codes 12 It is possible to identify each three-dimensional position coordinate from a set of coordinates.

As a method of shooting with a camera at two points or a plurality of points shown in FIG. 20, a method of shooting from a fixed viewpoint while partially overlapping the shooting fields of view using two shooting devices 2A and 2B of the same type is used. One imaging device 2A is moved between two viewpoints and is roughly classified into a method of photographing each. An example of the former is a method in which a plurality of surveillance cameras are provided on the ceiling and the barcode 12 is searched and decoded using each surveillance image. In addition, as an example of the latter, in addition to the method of artificially giving two shooting positions each time to perform measurement, a camera is mounted on a moving object such as an autonomous mobile robot or a mobile trolley, and a part of the shooting range is overlapped. There is a method of repeating the moving shooting while letting it move. In the latter case, the same barcode is photographed from two or more viewpoints, for example, three viewpoints, and the same barcode is identified from the barcode decoding information or the surrounding image information. It is possible to obtain the three-dimensional position coordinates in the global coordinate system from the set of local directional coordinate components.

In this way, by reading the bar code for the shot images taken at the plurality of positions and detecting the bar code position, for example, in a space such as a warehouse or a store, a plurality of bar codes 12 arranged in a wide range are separated from each other. It is possible to provide a bar code recognition device capable of detecting from the bar code and efficiently reading the bar code information. Further, it becomes possible to provide an article management system capable of obtaining information on the position of the barcode in space as well as reading the barcode information.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

Further, the process described in the above-described embodiment is a recording medium such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), a semiconductor memory, etc., as a program that can be executed by a computer. Can be provided to various devices by writing to. It is also possible to transmit it through a communication medium and provide it to various devices. The computer reads the program recorded on the recording medium or receives the program via the communication medium, and the operation is controlled by the program to execute the above-mentioned processing.

1 ... Bar code recognition device, 2 ... Bar code reading unit, 3 ... Display unit, 4 ... Input unit, 5 ... ROM, 6 ... CPU, 7 ... RAM, 8 ... Communication unit, 9 ... I / F unit.

Claims (4)

A reading means for shooting toward a shooting range including a code image to be read by a camera equipped with a pan / tilt function, and
An area detecting means for detecting a code area and a character area included in the code image from the code image captured by the reading means, and
A position detecting means for detecting the position of the code area based on the position of the code area in the code image and the pan / tilt information indicating the direction of the camera adjusted by the pan / tilt function.
In parallel with the first extraction means for extracting the first code information represented by the code image from the code area and the extraction of the first code information by the first extraction means, the code image represents from the character area. It has a second extraction means for extracting 2 code information, and has
The area detecting means adjusts the direction of the camera by the reading means at a plurality of different positions by the pan / tilt function, and detects the code area from a plurality of code images taken respectively.
The position detecting means of the code area is based on the position of the code area detected from the plurality of code images and the pan / tilt information indicating the orientation of the camera at the time of shooting at the plurality of different positions. A code recognition device that detects a position in a three-dimensional space . The code recognition device according to claim 1, further comprising an information integration means for integrating the first code information and the second code information. The code recognition device according to claim 2, wherein the information integrating means outputs either one of the first code information and the second code information extracted corresponding to the code image. The code recognition device according to claim 1, wherein the center of rotation of the pan / tilt of the camera having the pan / tilt function is common.

Images