JP2023031715A - Image processing program, image processing method, and image processing apparatus - Google Patents

Abstract

To improve operation efficiency in reading a bar-code.SOLUTION: A processing unit 1a is configured to: acquire a plurality of captured images captured successively by a camera 2 while a truck 3 loaded with a plurality of boxes passes in front of the camera 2; detect coordinates of a bar-code and a read value of the bar-code from each of the captured images; extracts coordinates of bar-codes in a pair of adjacent captured images having the same read value, out of the multiple captured images, as corresponding feature points; and output read values of the bar-codes arranged on surfaces of the boxes and the number of the bar-codes, on the basis of coordinates of the bar-codes detected from each of the multiple captured images, in a coordinate system of a panorama image to be obtained by synthesizing the multiple captured images based on the coordinates of the extracted corresponding feature points, and the read values of the bar-codes.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing program, an image processing method, and an image processing apparatus.

Barcodes are used to encode and display various types of information, and are widely used for product management and the like. For example, in Japan, a "JAN (Japanese Article Number) code" is widely used as a bar code indicating product identification information. The JAN code is used for managing products to be shipped and products to be sold.

Moreover, reading of bar codes has often been performed using a dedicated reader. Recently, however, it has become possible to relatively easily read a barcode from an image of the barcode captured with a general camera using an application for a PC (Personal Computer) or a smartphone.

As a technique related to barcodes, for example, an image processing apparatus has been proposed that makes searching for the barcode of a desired product more efficient by visually indicating the position of the barcode corresponding to the product information in the photographed image. As another example, any one of the sales product data of the product, the captured image data of the bar code acquired from the photographed image, and the synthesized image data obtained by synthesizing the sales product data and the captured image data. is proposed on the display surface of a product sales registration data processing device.

Japanese Patent Application Laid-Open No. 2020-86660 JP 2008-257462 A

By the way, a bar code such as a JAN code is, for example, attached to the surface of a box (cardboard box, etc.) in which products are packed. In addition, there is a case in which the bar code of each box is read for the purpose of product inspection or the like while such boxes are stacked on a trolley for transportation.

In such a case, for example, if a method of reading barcodes one by one using a handy reader is adopted, work efficiency is poor. For this reason, it is conceivable to photograph a plurality of barcodes in one image with a camera and automatically read the contents of the plurality of barcodes from the obtained image. However, if the space in which the box is to be transported is small and the distance between the camera and the box cannot be long, the operator must take multiple images so that all the barcodes are captured. There is a problem that it takes time to shoot.

In one aspect, an object of the present invention is to provide an image processing program, an image processing method, and an image processing apparatus capable of streamlining barcode reading work.

In one proposal, a plurality of boxes each containing an item and having a bar code indicating information about the contained item placed on the surface of each box are loaded in the computer, and the trolley is positioned over the camera. A plurality of captured images continuously captured by the camera while passing across the front are acquired, the coordinates of the barcode and the read value of the barcode are detected from each of the plurality of captured images, and a plurality of The coordinates of the barcode having the same read value between the first pair of captured images adjacent to each other among the captured images are extracted as corresponding feature points between the first pair of captured images, and the coordinates of the extracted corresponding feature points are extracted as Based on the barcode coordinates detected from each of the plurality of photographed images in the coordinate system of the panorama image obtained by synthesizing the plurality of photographed images, and the reading value of the barcode, the coordinates of the plurality of boxes are determined. An image processing program is provided that causes the processing to output readings and numbers of barcodes placed on the surface.

Another proposal provides an image processing method in which a computer executes processing based on the above image processing program.
Furthermore, one proposal provides an image processing apparatus that executes processing based on the above image processing program.

On one side, it is possible to improve the efficiency of barcode reading work.

1A and 1B are diagrams illustrating a configuration example and a processing example of an image processing apparatus according to a first embodiment; FIG. It is a figure which shows the structural example of the shipping inspection system which concerns on 2nd Embodiment. It is a figure which shows the hardware structural example of PC. It is a figure which shows the example of the installation state of a camera. 3 is a diagram illustrating a configuration example of processing functions provided in a PC; FIG. FIG. 10 is a diagram showing basic processing of image synthesis in the horizontal direction; FIG. 11 is a first diagram showing an example of processing for selecting the same point; FIG. 11 is a second diagram showing an example of selection processing for the same point; FIG. 10 is a diagram showing an example of synthesis processing when no JAN code with the same read value is detected; 4 is a diagram showing the relationship between the viewing angle of the camera and the shooting range; FIG. FIG. 4 is a diagram showing the relationship between JAN code size and image size; It is a figure which shows the calculation method of the overlapping width|variety between picked-up images. FIG. 10 is a diagram showing a process of detecting and calculating the vertical size of JAN code; FIG. 10 is a diagram for explaining JAN code identification processing; FIG. 10 is a diagram for explaining a determination threshold determination method for JAN code identification; It is an example of a flow chart showing the overall processing of a PC. 7 is an example of a flowchart showing a procedure of shift width calculation processing; FIG. 11 is an example of a flowchart showing a procedure of overlapping width calculation processing; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram illustrating a configuration example and a processing example of an image processing apparatus according to the first embodiment. The image processing apparatus 1 shown in FIG. 1 includes a processing section 1a. The processing unit 1a is, for example, a processor (not shown) included in the image processing apparatus 1. FIG. A camera 2 is also connected to the image processing device 1 , and the image processing device 1 can receive images captured by the camera 2 .

A truck 3 loaded with a plurality of boxes passes across in front of the camera 2. - 特許庁The boxes loaded on the truck 3 are, for example, cardboard boxes, and each box contains a product. In addition, a bar code indicating information on the contained product is arranged on each surface of the box. A plurality of boxes are loaded on the truck 3 so that the surface on which the bar codes are arranged faces the camera 2, and the truck 3 in such a state moves across the front of the camera 2.例文帳に追加In the example of FIG. 1, the cart 3 moves from right to left with respect to the imaging direction of the camera 2 .

The camera 2 continuously takes a plurality of shot images while the carriage 3 is passing across in front of the camera 2 and transmits them to the image processing device 1.例文帳に追加At this time, the camera 2 takes an image so that the boxes loaded on the truck 3 are reflected. The processing unit 1a of the image processing device 1 acquires a plurality of captured images thus captured (step S1). Then, the processing unit 1a executes the following processing for outputting the read values of the barcodes arranged on each of the stacked boxes using the acquired plurality of photographed images.

The processing unit 1a detects a barcode from each of the plurality of captured images (step S2). At this time, the processing unit 1a detects the coordinates in the photographed image of the barcode and the read value of the barcode.

Next, the processing unit 1a extracts each coordinate of the barcode having the same reading value between the adjacent pair of captured images among the acquired plurality of captured images as the corresponding feature point between the pair of captured images. (Step S3). Through this processing, corresponding feature points of each captured image included in the pair are extracted for each captured image pair.

Next, the processing unit 1a creates a panorama image by synthesizing a plurality of photographed images based on the extracted coordinates of the corresponding feature points. However, the panorama image does not have to be actually created, and at least the coordinates of the barcode detected in each captured image should be converted to the coordinates of the coordinate system of the panorama image. Based on the coordinates of each barcode in the coordinate system of the panorama image and the read value of each barcode, the processing unit 1a calculates the read values and the number ( number of bar codes with the same read value) and output (step S4).

By such processing, the contents of the barcodes arranged on the stacked boxes are automatically read, so that the barcode reading work can be made more efficient.
An example of a captured image captured by the camera 2 is shown on the right side of FIG. In FIG. 1, as an example, captured images P1, P2, . . . are captured in order. Also, from the photographed image P1, a barcode 4a1 with a read value of "0001" and a barcode 4b with a read value of "0002" are detected. A barcode 4a2 with a read value of "0001" and a barcode 4c with a read value of "0003" are detected from the captured image P2.

In this case, between the captured image pair including the captured images P1 and P2, the coordinates of the barcodes 4a1 and 4a2 having the same read value are extracted as corresponding feature points. By extracting the corresponding feature points, the processing unit 1a determines the shift width of the photographed image P2 when synthesizing the photographed image P2 with the photographed image P1 based on the result of comparison between one coordinate and the other coordinate. (width of deviation) can be calculated. The processing unit 1a can synthesize the synthesized image P0 by transforming the coordinates of at least one of the captured images P1 and P2 into the coordinates of the coordinate system of the synthesized image P0 based on the shift width. Note that the synthesized image P0 is a part of the panoramic image described above.

The barcodes 4a, 4b, and 4c are shown in the composite image P0 shown in FIG. Among these, the barcode 4a corresponds to the barcodes 4a1 and 4a2. For example, the processing unit 1a detects that each barcode with read values "0001", "0002", and "0003" is detected one by one from the results of reading the barcodes 4a, 4b, and 4c on the composite image P0. The processing result shown can be output.

Here, a box such as a cardboard box is rarely drawn with fine patterns on its surface and has a simple outer shape. For this reason, when a general feature point extraction method is used, the feature points cannot be extracted accurately, and as a result, the extraction accuracy of the corresponding feature points between the captured image pairs is lowered. On the other hand, in the present embodiment, the coordinates of barcodes having the same read value between the captured image pairs are extracted as the corresponding feature points. This makes it possible to accurately extract corresponding feature points. As a result, it is possible to improve the accuracy of panorama image creation or the accuracy of coordinate conversion with respect to the coordinate system of the panorama image. Finally, the barcode reading results (read value and number) can be output accurately.

As described above, the image processing apparatus 1 according to the first embodiment can accurately perform automatic barcode reading using a plurality of captured images continuously captured by the camera 2 . As a result, the barcode reading work can be made more efficient.

[Second embodiment]
Next, a product shipment inspection system including the image processing apparatus 1 shown in FIG. 1 will be described.
FIG. 2 is a diagram showing a configuration example of a shipping inspection system according to the second embodiment. This shipment inspection system supports inspection work at the time of shipment of products packed in corrugated cardboard boxes, and includes cameras 10a, 10b, 10c, . The PC 100 is an example of the image processing apparatus 1 in FIG. 1, and the cameras 10a, 10b, 10c, . . . are examples of the camera 2 in FIG.

The cameras 10a, 10b, 10c, . As will be described later, the cameras 10a, 10b, 10c, . . . shoot images continuously at regular time intervals. In addition, photographing operations are synchronized among the cameras 10a, 10b, 10c, . . .

The illumination 20 irradiates light onto the range photographed by the cameras 10a, 10b, 10c, . . . to clarify the photographed image. A plurality of lights 20 may be installed, or the lights 20 may be connected to the PC 100 and light irradiation operations may be controlled from the PC 100 .

The PC 100 reads the barcode displayed on the surface of the cardboard box from the images captured by the cameras 10a, 10b, 10c, . Note that the PC 100 is an example of an information processing apparatus that executes such processing, and the processing of the PC 100 may be executed by another type of information processing apparatus such as a server computer.

FIG. 3 is a diagram showing a hardware configuration example of a PC. The PC 100 is realized as a computer as shown in FIG. 3, for example. The PC 100 shown in FIG. 3 includes a processor 101, a RAM (Random Access Memory) 102, a HDD (Hard Disk Drive) 103, a GPU (Graphics Processing Unit) 104, an input interface (I/F) 105, a reader 106, a network interface ( I/F) 107 and communication interface (I/F) 108 .

A processor 101 centrally controls the entire PC 100 . The processor 101 is, for example, a CPU (Central Processing Unit), MPU (Micro Processing Unit), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), or PLD (Programmable Logic Device). Also, the processor 101 may be a combination of two or more of CPU, MPU, DSP, ASIC, and PLD.

A RAM 102 is used as a main storage device of the PC 100 . The RAM 102 temporarily stores at least part of an OS (Operating System) program and application programs to be executed by the processor 101 . Various data necessary for processing by the processor 101 are stored in the RAM 102 .

HDD 103 is used as an auxiliary storage device for PC 100 . The HDD 103 stores an OS program, application programs, and various data. Other types of non-volatile storage devices such as SSDs (Solid State Drives) can also be used as auxiliary storage devices.

A display device 104 a is connected to the GPU 104 . The GPU 104 causes the display device 104a to display an image according to instructions from the processor 101 . Examples of the display device 104a include a liquid crystal display and an organic EL (ElectroLuminescence) display.

An input device 105 a is connected to the input interface 105 . The input interface 105 transmits signals output from the input device 105 a to the processor 101 . The input device 105a includes a keyboard, pointing device, and the like. Pointing devices include mice, touch panels, tablets, touch pads, trackballs, and the like.

A portable recording medium 106 a is attached to and detached from the reading device 106 . The reading device 106 reads data recorded on the portable recording medium 106 a and transmits the read data to the processor 101 . As the portable recording medium 106a, there are an optical disk, a semiconductor memory, and the like.

A network interface 107 transmits and receives data to and from another device via a network 107a.
A communication interface 108 transmits and receives data to and from the cameras 10a, 10b, 10c, .

The processing functions of the PC 100 can be realized by the hardware configuration as described above.
In the following description, the cameras 10a, 10b, 10c, .

FIG. 4 is a diagram showing an example of an installation state of cameras. 4A shows a side view of the installation location of the camera 10, and FIG. 4B shows a plan view of the installation location of the camera 10. FIG.
In the present embodiment, shipping inspection of products is performed in a state where the cardboard boxes 32 of the products are loaded on the cart 31 . As shown in FIG. 4, a plurality of cardboard boxes 32 can be arranged side by side on the carriage 31 in the lateral direction (direction of travel of the carriage 31). A plurality of cardboard boxes 32 can also be stacked on the carriage 31 in the vertical direction.

A label 33 printed with a JAN code indicating product identification information is attached to the surface of each cardboard box 32 . A JAN code is a bar code symbol that encodes product identification information. The cardboard boxes 32 are loaded on the truck 31 so that the labels 33 are oriented in the same horizontal direction with respect to the traveling direction of the truck 31 . In this embodiment, as an example, as shown in FIG. 4, the cardboard boxes 32 are stacked so that the label 33 faces the left side (lower side in FIG. 4B) in the traveling direction. In such a state, shipping inspection is performed by reading the JAN code printed on the label 33 of each corrugated cardboard box 32 .

In such a case, for example, if a method of reading JAN codes one by one using a handy barcode reader is adopted, work efficiency is poor. On the other hand, the work efficiency is improved by a method in which a plurality of JAN codes are captured in one image by the camera 10, and the plurality of JAN codes are automatically read from the obtained image by the processing of the PC 100. can be enhanced.

However, in order to fit the JAN codes of all the cardboard boxes 32 loaded on the truck 31 into one image, a wide space is required in the horizontal direction with respect to the traveling direction of the truck 31 . It is rare that such a large space can be prepared for actual shipping inspection work. For example, as shown in FIG. 4(B), there are cases where only a shooting distance of about 30 cm can be taken between the camera 10 and the shooting target. For this reason, a plurality of images are photographed so as to cover the photographing range including all the cardboard boxes 32 loaded on the cart 31, and all the JAN codes are automatically read from the photographed images. desired.

In this embodiment, in order to achieve such an object, a plurality of cameras 10 are arranged in the vertical direction, and these cameras 10 cover the vertical imaging range. Furthermore, by causing the carriage 31 to pass in front of the camera 10 and continuously photographing images during its passage, the photographing range in the lateral direction is covered.

In FIG. 4, four cameras 10a to 10d are installed as an example. These cameras 10a to 10d are installed at regular intervals in the vertical direction. The cameras 10a to 10d are installed in the left direction with respect to the traveling direction of the truck 31, and the truck 31 pushed by the worker 34 passes in front of the cameras 10a to 10d. In this case, the carriage 31 moves from the right to the left with respect to the photographing direction of the cameras 10a to 10d. While the carriage 31 is moving, each of the cameras 10a to 10d continuously captures images at regular time intervals.

In this way, a plurality of photographed images are obtained in each of the vertical and horizontal directions. The PC 100 synthesizes a plurality of obtained captured images into a panorama image, and outputs the reading value and the number of JAN codes on the panorama image as a processing result. As a result, the JAN codes of all cardboard boxes 32 on the cart 31 are automatically read.

Note that continuous shooting of images is performed by, for example, moving image shooting. In this case, captured images (frames) at regular time intervals are extracted from the captured moving images. As another example, each camera 10 may capture still images at regular time intervals under the control of the PC 100 . On the other hand, the cameras 10 arranged in the vertical direction are synchronously photographed. Therefore, an image shot at a certain shooting time is obtained from each of the cameras 10 arranged in the vertical direction.

FIG. 5 is a diagram showing a configuration example of processing functions provided in a PC. As shown in FIG. 5, the PC 100 includes a storage section 110, an image receiving section 121, a JAN code reading section 122, an image synthesizing section 123 and a result output processing section .

The storage unit 110 is a storage area of a storage device included in the PC 100, such as the RAM 102 and the HDD 103, for example. The storage unit 110 stores camera setting information 111, JAN code information 112, image information 113, and JAN code reading information 114. FIG.

In the camera setting information 111, information regarding the cameras 10a, 10b, 10c, . . . is set in advance. For example, in the camera setting information 111, the height of each of the cameras 10a, 10b, 10c, .

In the JAN code information 112, the vertical and horizontal sizes of the JAN code on the label attached to the cardboard box 32 and the shortest distance between the JAN codes are set in advance. The shortest distance between JAN codes indicates the distance between the center points of the printed areas of the JAN codes when the JAN codes of adjacent cardboard boxes 32 are closest when the cardboard boxes 32 are stacked.

The image information 113 accumulates image data of each captured image (still image) captured by the cameras 10 a, 10 b, 10 c, . . . and received by the PC 100 .
The JAN code reading information 114 stores the position (coordinates on the image) and reading value of the JAN code detected from each captured image.

The processes of the image receiving unit 121, the JAN code reading unit 122, the image synthesizing unit 123, and the result output processing unit 124 are realized, for example, by the processor 101 executing a predetermined program.

The image receiving unit 121 receives data of captured images from the cameras 10a, 10b, 10c, .
The JAN code reading unit 122 detects the image area of the JAN code from each captured image, and reads the numerical value (read value) indicated by the JAN code. The JAN code reading unit 122 associates the position (coordinates) of the JAN code in the captured image with the read value of the JAN code and registers them in the JAN code reading information 114 .

The image synthesizing unit 123 joins and synthesizes the photographed images to create a panoramic image.
The result output processing unit 124 outputs the read value and the number of JAN codes on the created panorama image as a processing result together with the panorama image.

Next, synthesizing processing of panoramic images will be described. First, with reference to FIGS. 6 to 9, the process of synthesizing images captured in the horizontal direction will be described. Images captured in the horizontal direction refer to images continuously captured by the same camera 10 .

FIG. 6 is a diagram showing basic processing of image composition in the horizontal direction.
In general, it is necessary to detect the same points (corresponding points) between consecutive images in order to synthesize a panoramic image. When the same point is detected, it is possible to calculate the overlap width when combining the images based on the coordinates of the same point in each image.

As a method for detecting the same points between images, for example, a method of extracting feature points from each image using a feature point extraction algorithm such as SIFT (Scaled Invariance. Feature Transform) or SURF (Speed Up Robust Feature) is considered. be done. However, the corrugated cardboard box to be photographed in the present embodiment has few characteristic patterns on the surface and has a simple outer shape. For this reason, feature points that are effective as corresponding points are difficult to detect, and mismatches between corresponding points are likely to occur.

As another method, it is conceivable to attach markers serving as characteristic points on the surface of the carriage 31 in parallel in the lateral direction. However, it is necessary to attach the marker to the existing truck 31 or to introduce a new truck 31 with the marker, which increases the introduction cost. In addition, since it is unknown to which existing carriage 31 the marker can be attached, versatility is also lacking.

Therefore, in the present embodiment, JAN codes (barcode symbols of JAN codes) are extracted as feature points. Then, JAN codes having at least the same read value between images are detected as the same point.

As described above, the label 33 on which the JAN code is printed is attached to the surface of the cardboard box in which the product is packed. As shown in the upper part of FIG. 6, the JAN code reader 122 of the PC 100 detects the JAN code area 35 in the label 33 from the captured image by image recognition technology. The JAN code reading unit 122 outputs, for example, the coordinates of the upper left edge point 36 in the area 35 as the position of the JAN code on the captured image. Along with this, the JAN code reading unit 122 outputs the read value of the JAN code.

The lower part of FIG. 6 shows an example of the Nth shot image P11 and the (N+1)th shot image P12 in the horizontal direction.
The JAN code 201a with the read value "JAN001" and the JAN code 202a with the read value "JAN002" appear in the Nth shot image P11. In addition, the JAN code reader 122 acquires the position and read value for both the JAN codes 201a and 202a.

In the (N+1)th photographed image P12, the JAN code 201b of the read value "JAN001", the JAN code 202b of the read value "JAN002", and the JAN code 203b of the read value "JAN003" appear. Of these, the JAN code reader 122 acquires the positions and reading values of the JAN codes 201b and 203b. On the other hand, the JAN code 202b is only partly shown in the photographed image P12, and the JAN code reading unit 122 has not acquired the position and read value.

The image synthesizing unit 123 basically selects JAN codes having the same read values between the photographed image P11 and the photographed image P12 from among the JAN codes whose positions and read values are obtained from the photographed images P11 and P12. Detect as In the example of FIG. 6, the JAN code 201a on the captured image P11 and the JAN code 201b on the captured image P12 are detected as the same point.

Let the coordinates of the JAN code 201a be (x1, y1) and the coordinates of the JAN code 201b be (x2, y2). In this case, the image synthesizing unit 123 calculates the difference |x2−x1| (absolute value of (x2−x1)) between the X coordinate x2 of the JAN code 201b and the X coordinate x1 of the JAN code 201a as the captured image for the captured image P11. It is calculated as the horizontal shift width of P12. Then, the image synthesizing unit 123 can create a synthesized image P13 by synthesizing the photographed image P12 at a position shifted to the right by |x2−x1| from the left end of the photographed image P11.

Note that when the carriage 31 moves from the left side to the right side with respect to the photographing direction, the image synthesizing unit 123 places the (N+1)th photographed image at a position shifted to the left by |x2−x1| from the right end of the Nth photographed image. A synthesized image is created by synthesizing the captured images of the

FIG. 7 is a first diagram showing an example of processing for selecting the same point. The same point between captured images is always detected between adjacent captured images. Even if JAN codes with the same reading value are detected between discontinuous captured images, those JAN codes are not selected as the same point (same JAN code).

In the example of FIG. 7, the JAN code 211a with the reading value "JAN001" is detected in the Nth shot image P21, and the JAN code 211b with the reading value "JAN001" is detected in the (N+1)th shot image P22. there is Also, the JAN code 212a with the reading value "JAN001" is detected in the (N+10)th photographed image P23, and the JAN code 212b with the reading value "JAN001" is detected in the (N+11)th photographed image P24.

In this case, the image synthesizing unit 123 selects the JAN codes 211a and 211b as the same point, and selects the JAN codes 212a and 212b as the same point. However, the image synthesizing unit 123 does not select, for example, the JAN codes 211b and 212a that are not detected between discontinuous captured images as the same point. As a result, for example, it is possible to prevent the captured image P22 and the captured image P23 from being joined together and the captured images from the (N+2)th to the (N+9)th images to be lost.

FIG. 8 is a second diagram showing an example of selection processing for the same point. A plurality of JAN codes having the same read value may be detected from each of adjacent captured images. In this case, it is necessary to determine which JAN code of each photographed image is selected as the same point in order to enable accurate synthesis processing.

In the example of FIG. 8, the JAN codes 221a and 222a of the read value "JAN001" and the JAN code 223a of the read value "JAN002" are detected in the Nth shot image P31. Also, in the (N+1)th photographed image P32, the JAN codes 221b and 222b of the reading value "JAN001" and the JAN code 223b of the reading value "JAN002" are detected.

In this case, the JAN codes 221a and 221b are actually the same JAN code, and the JAN codes 222a and 222b are the same JAN code. Therefore, the combination of the JAN codes 221a and 221b or the combination of the JAN codes 222a and 222b must be detected as the same point for accurate synthesis processing. However, if the combination of the JAN codes 221a and 222b or the combination of the JAN codes 222a and 221b are detected as the same point, the correct synthesis processing cannot be performed, and there is a possibility that other JAN codes will be lost due to the synthesis processing. be.

Therefore, when a plurality of JAN codes having the same read value are detected from the same photographed image, the image synthesizing unit 123 selects the JAN code with the largest X coordinate from among them to search for the same point. conduct. In the case of FIG. 8, the JAN code 222a is selected from the captured image P31, the JAN code 222b is selected from the captured image P32, and the selected JAN codes 222a and 222b are detected as the same point. On the other hand, for example, the JAN codes 221a and 222b are determined not to be the same point.

Note that when the carriage 31 moves from the left side to the right side with respect to the photographing direction, the image synthesizing unit 123 selects the JAN code with the smallest X coordinate from among a plurality of JAN codes with the same read value, Search for the same point.

As another method, the image synthesizing unit 123 selects the maximum X coordinate of the JAN codes 221a and 222a having the same read value from the temporally front captured image P31 instead of the adjacent captured images P31 and P32. , the JAN code 222a may be selected. In this case, the image synthesizing unit 123 selects a JAN code whose Y coordinate is within a certain range with respect to the JAN code 222a from among the JAN codes 221b and 222b detected from the captured image P32 on the rear side in terms of time. Detect as the same point.

FIG. 9 is a diagram showing an example of synthesizing processing when JAN codes with the same read value are not detected. As described above, the image synthesizing unit 123 detects JAN codes with the same read value as the same point between successive captured images. However, it is not always the case that a JAN code with the same reading value is detected between successive captured images.

In the example of FIG. 9, the JAN code 231a with the read value "JAN001" and the coordinates (x1, y1) is detected from the Nth photographed image P41. Also, the JAN code 231b of the read value "JAN001" and the coordinates (x2, y2) is detected from the (N+1)th photographed image P42. These JAN codes 231a and 231b are detected as the same point, and the shift width between the photographed images P41 and P42 is calculated as |x2-x1|.

Also, the JAN code 232b of the read value "JAN003" and coordinates (x3, y3) is detected from the (N+1)th photographed image P42. Further, the JAN code 232c of the read value "JAN003" and the coordinates (x4, y4) is detected from the (N+2)th photographed image P43. These JAN codes 232b and 232c are detected as the same point, and the shift width between the photographed images P42 and P43 is calculated as |x4-x3|.

On the other hand, in the (N+3)th shot image P44, the JAN code 232d that is actually the same as the JAN code 232c appears. However, the JAN code 232d is only partly shown in the captured image P44 and is not detected by the image synthesizing section 123. FIG. Furthermore, the JAN code is not shown in the (N+4)th photographed image P45. In such a case, the shift width between the photographed images P43 and P44 and the shift width between the photographed images P44 and P45 cannot be calculated, and the photographed images P44 and P45 cannot be synthesized.

Here, for example, unless the operator 34 pushing the carriage 31 intentionally stops or returns, it is assumed that the carriage 31 moves at a substantially constant speed. Therefore, when the image synthesizing unit 123 cannot detect a JAN code with the same reading value between a certain captured image and the next captured image, the shift width calculated between the most recent captured images is added to the corresponding captured image. It is used as it is as the shift width between

In the example of FIG. 9, the shift width |x4-x3| between the photographed images P42 and P43 is used as the shift width between the photographed images P43 and P44. Further, the shift width |x4-x3| between the photographed images P42 and P43 is also used as the shift width between the photographed images P44 and P45. As a result, not only the photographed images P41 to P43 but also the photographed images P44 and P45 can be synthesized to create a synthetic image P46.

With this method, there is a possibility that the calculation accuracy of the shift width will be lowered and the image synthesis accuracy will be degraded. However, for shipping inspection, it is sufficient to know the read value and the number of JAN codes, so the image synthesis accuracy of the portion where the JAN code is not shown is not so important. By the above method, it is possible to obtain necessary and sufficient detection accuracy of the same point.

Synthesis processing of vertically captured images will now be described with reference to FIGS. 10 to 13. FIG. The captured images in the vertical direction refer to images captured by the cameras 10a, 10b, 10c, . . . at the same time.

As for the captured images in the vertical direction, a method of synthesizing successive captured images (where the cameras 10 are adjacent) can be considered by using JAN codes with the same read value detected from the captured images as feature points. In the synthesis of the photographed images in the horizontal direction, the appearance of the JAN code can be adjusted by adjusting the moving speed of the carriage 31. In the vertical direction, however, the interval between the cameras 10 is fixed. Therefore, when attempting to capture the JAN code with the same reading value in both vertically adjacent photographed images, the length of overlap between the photographing ranges in the vertical direction of each camera 10 becomes relatively large. It becomes necessary to reduce the difference in height. Empirically, the photographed images cannot be synthesized with high accuracy unless the photographing ranges overlap by ⅔ or more. However, if the difference in height between the cameras 10 is reduced, the number of installed cameras 10 must be increased, resulting in an increase in component cost. Therefore, it is difficult to use the method of detecting the JAN code as a feature point for synthesizing vertically captured images.

Therefore, the image synthesizing unit 123 calculates the overlapping width between the shot images when synthesizing the consecutive shot images without using the above method. In the method described below, if at least one JAN code is detected from each photographed image in the vertical direction regardless of the reading value, the overlapping width can be calculated with high accuracy. As a result, the number of cameras 10 is reduced as much as possible, and the device cost is reduced.

The image synthesizing unit 123 first obtains the distance between the camera 10 and the surface to be photographed (the surface of the cardboard box). Here, the distance between the camera 10 and the surface to be photographed is generally determined when the camera 10 is installed. However, the distance between the camera 10 and the surface to be photographed may vary depending on the size of the cardboard boxes loaded on the cart 31 and how they are stacked on the cart 31 . Furthermore, since the truck 31 is pushed by a person, the distance between the truck 31 and the camera 10 is not always the same.

Therefore, based on the number of pixels in the vertical direction (vertical size) of the JAN code area in the captured image and the preset camera setting information 111, the image synthesizing unit 123 determines the distance between the camera 10 and the surface to be captured. Estimate the distance of

FIG. 10 is a diagram showing the relationship between the viewing angle of the camera and the shooting range. In FIG. 10, the distance D indicates the distance (unit: mm) between the camera 10 that captured the captured image and the surface 37 to be captured. The viewing angle θ indicates the viewing angle (vertical viewing angle) of the camera 10 with respect to the vertical direction. This viewing angle θ is preset in the camera setting information 111 . The height H indicates the length (unit: mm) in the vertical direction of the shooting range (the range appearing in the shot image) on the shooting target plane 37 . The height h refers to the half length of the height H (unit: mm). At this time, the height h is represented by the following formula (1).
h=D×tan(θ/2) (1)
FIG. 11 is a diagram showing the relationship between the JAN code size and the image size. In FIG. 11, the vertical size IH indicates the number of pixels in the vertical direction of the captured image P51. This vertical size IH is preset in the camera setting information 111 . The vertical size PQH indicates the number of pixels in the vertical direction of the area of the JAN code 38 in the captured image P51. The height QH indicates the vertical length (unit: mm) of the area of the JAN code 38 on the imaging target plane. At this time, the relationship between the vertical sizes IH, PQH and the heights H, QH is represented by the following equation (2).
PQH:QH=IH:H (2)
Then, the vertical size PQH of the JAN code is expressed by the following equation (3) from this equation (2) and the above equation (1).
PQH=(QH×IH)/H
=(QH×IH)/(2×D×tan(θ/2)) (3)
Therefore, by detecting the vertical size PQH of the JAN code 38 in the captured image P51, the image synthesizing unit 123 can calculate the distance D from the camera 10 to the JAN code 38 using the following equation (4). can.
D=(QH×IH)/(2×tan(θ/2))×1/PQH (4)
FIG. 12 is a diagram showing a method of calculating the overlapping width between captured images. With reference to FIG. 12, a method of calculating the overlapping width between the captured images when synthesizing the captured image by the camera 10a and the captured image by the camera 10b will be described.

In FIG. 12, the heights (unit: mm) of the cameras 10a and 10b from the ground surface on which the truck 31 moves are assumed to be SH1 and SH2, respectively. Also, the viewing angles θ of the cameras 10a and 10b are the same. Further, the distances (unit: mm) from the cameras 10a and 10b to the imaging target surface 37 are assumed to be D1 and D2, respectively. In FIG. 12, the surface to be photographed by the camera 10a and the surface to be photographed by the camera 10b are represented as the same surface for the sake of easy understanding of the description, but they may actually be different. Therefore, the distance D1 and the distance D2 may not be equal.

The image synthesizing unit 123 estimates the vertical shooting range on the shooting target plane for each of the cameras 10a and 10b. This makes it possible to calculate the overlap width in which the respective imaging ranges overlap.

In FIG. 12, the upper end position and the lower end position of the imaging range of the camera 10a on the imaging target plane of the camera 10a are set to YH1 and YL1, respectively (unit: mm). Also, let h1 be the height that is half the vertical length of the photographing range (unit: mm). On the other hand, the upper end position and the lower end position of the photographing range of the camera 10b on the photographing target plane of the camera 10b are set to YH2 and YL2, respectively (unit: mm). Also, let h2 be the height that is half the vertical length of the photographing range (unit: mm). In this case, the upper end position YH1 of the imaging range of the camera 10a and the lower end position YL2 of the imaging range of the camera 10b are represented by the following equations (5) and (6), respectively. Therefore, the overlapping width W (unit: mm) of these photographing ranges is calculated by the following formula (7).
YH1=SH1+h1 (5)
YL2=SH2-h2 (6)
W=YH1-YL2 (7)
The heights h1 and h2 are expressed by the following formulas (8) and (9), respectively, using the above formula (1).
h1=D1×tan(θ/2) (8)
h2=D2×tan(θ/2) (9)
Here, the vertical size IH of the photographed image, the height h indicating half the length of the photographing range in the vertical direction, the overlap width W, the overlap width PW (unit: pixel) between the photographed images at the time of synthesis, and the photographing A relationship such as the following formula (10) holds with the range overlap width W (=YH1-YL2). Therefore, the image synthesizing unit 123 can calculate the overlapping width PW between the captured images using the following formula (11).
IH: 2×h=PW: (YH1-YL2) (10)
PW=(IH×(YH1−YL2))/(2×h) (11)
As the height h in Equation (11), for example, an average value between the height h1 corresponding to the camera 10a and the height h2 corresponding to the camera 10b can be used.

As described above, the image synthesizing unit 123 detects the vertical size PQH of the JAN code in each photographed image that is continuous in the vertical direction. Based on the detected vertical size PQH, and the vertical size IH, height QH, and viewing angle θ set in the camera setting information 111, the image synthesizing unit 123 calculates the distance from the shooting target plane for each captured image. Calculate D.

Further, the image synthesizing unit 123 calculates the upper end position and the lower end position of the photographing range for each photographed image based on the distance D calculated for each photographed image and the height of each camera, and based on this calculation result, An overlap width PW between captured images is calculated. Finally, based on the overlapping width PW between the vertically continuous captured images, the image synthesizing unit 123 divides the upper image by (vertical size IH of the captured image - overlapping width PW) with respect to the lower image. A synthesized image is created by synthesizing at the position shifted upward.

By the way, in the overlapping width calculation process described above, the distance D from the camera to the surface to be photographed is calculated based on the vertical size PQH of the JAN code in the photographed image. At this time, if a plurality of JAN codes are detected from the captured image, for example, the average value of the vertical size of each JAN code is substituted as the vertical size PQH in equation (4). However, the JAN code reading unit 122 can detect the area of the JAN code and read its contents even if only a part of the JAN code is shown in the photographed image in the vertical direction. Therefore, if the vertical size of such a JAN code is used in the calculation of Equation (4), the distance D cannot be calculated accurately.

FIG. 13 is a diagram showing the process of detecting and calculating the vertical size of the JAN code. As shown in the upper part of FIG. 13, for example, it is assumed that the JAN code reading unit 122 detects seven JAN codes from the photographed image P52. Of these, the JAN code shown in the upper left and the JAN code shown in the lower left are only partially shown in the photographed image P52 in the vertical direction. is detected and the content of the JAN code is read.

When calculating the vertical size PQH of the JAN code corresponding to the captured image P52, the image synthesizing unit 123 calculates an average value based on the vertical sizes of a plurality of JAN codes detected from the captured image P52, and calculates the average value. Used as vertical size PQH. However, if the vertical size of the JAN code appearing in the upper left and lower left of the photographed image P52 is used for calculating such an average value, the calculation accuracy of the distance D by Equation (4) deteriorates. As a result, the calculation accuracy of the overlapping width W between the captured images also deteriorates.

Therefore, the image synthesizing unit 123 calculates a vertical size histogram of the detected JAN code, as shown in the lower part of FIG. The image synthesizing unit 123, for example, excludes from the average value calculation the vertical size that is apart from the vertical size at which the frequency peaks by a certain value or more. In the histogram example of FIG. 13, the frequency peaks in the vertical size range of 60 to 69 pixels. For example, if the distance for exclusion is 20 pixels, the range of 40 pixels or less becomes the exclusion range from the average value calculation, and the vertical size of 40 pixels or less is not used in the average value calculation. As a result, it is possible to prevent the occurrence of a situation in which the calculation accuracy of the distance D is deteriorated.

As an actual process, for example, the image synthesizing unit 123 uses the JAN codes detected from all the captured images in the horizontal direction and the vertical direction to calculate a histogram of the vertical sizes of the JAN codes. The image synthesizing unit 123 uses the above-described procedure to determine, from the average value calculation, an area of the number of pixels separated by a certain value or more from the vertical size at which the frequency peaks as an out-of-calculation range. After that, when calculating the distance D for a certain captured image, if a plurality of JAN codes are detected from the captured image, the image synthesizing unit 123 excludes the JAN code whose vertical size is included in the exclusion range, The distance D is calculated using the average value of the vertical sizes of the remaining JAN codes as the vertical size PQH.

By the procedure described above, the shift widths corresponding to all pairs of photographed images consecutive in the horizontal direction are calculated, and the overlap widths corresponding to all pairs of photographed images consecutive in the vertical direction are calculated. Subsequently, the image synthesizing unit 123 converts the coordinates of each pixel of each captured image into coordinates of the coordinate system of the panorama image (composite image) using the shift width and overlap width calculation results.

Here, the origin of the coordinate system of the image is assumed to be the lower left. Further, as in the present embodiment, when the cart 31 moves from the right side to the left side with respect to the photographing direction of the camera 10, the origin of the coordinate system of the panorama image is the first photographed image in the horizontal and vertical directions. It is set to the origin of (photographed image at left end and bottom end).

Conversion formulas for converting the coordinates of each pixel of the captured image into the coordinates of the coordinate system of the panorama image are given by the following formulas (12) and (13).

Equation (12) is a transformation equation for transforming the X coordinate p _s x of the pixel of the sth captured image into the X coordinate wx in the coordinate system of the panorama image. H _k in Equation (12) indicates the feature point distance (shift width) between the k-th captured image and the (k+1)-th captured image.

Expression (13) is a conversion expression for converting the Y coordinate p _n y of the pixel of the image captured by the n-th camera from the bottom into the Y coordinate wy in the coordinate system of the panorama image. V _k in Equation (13) represents the feature point distance (overlapping width) between the image captured by the k-th camera and the image captured by the (k+1)-th camera.

The image synthesizing unit 123 synthesizes each pixel value of the overlapping area between the captured images after the conversion by the above formulas (12) and (13) by a pixel value synthesizing method such as the alpha blending method, thereby obtaining a panorama image. Create an image. In addition, the position (coordinates) of the JAN code detected from the captured image is also transformed by the coordinate transformation described above. As will be described later, duplication of the JAN code is eliminated based on the coordinates of the converted JAN code, and the read value and number of the JAN code are output.

Note that, in the present embodiment, since a synthesized panoramic image is output, coordinate transformation using equations (12) and (13) is performed for all pixels of each captured image. However, it is also possible to output only the read value and the number of JAN codes without outputting the panorama image. In this case, only the position of the JAN code detected from each captured image should be subjected to coordinate transformation using equations (12) and (13).

FIG. 14 is a diagram for explaining JAN code identification processing. Coordinates of the JAN code detected from each photographed image are converted into coordinates of the coordinate system of the panorama image by coordinate conversion using the above equations (12) and (13). As a result, identical JAN codes in adjacent captured images are ideally converted to identical coordinates. The result output processing unit 124 eliminates duplication of JAN codes by deleting one of the JAN codes converted to the same coordinates from the JAN code reading information 114 . Between the photographed images adjacent in the horizontal direction, for example, the JAN code in the temporally front photographed image is deleted. Between vertically adjacent captured images, for example, the JAN code in the lower captured image is deleted.

However, in reality, it is rare that the coordinates of the same JAN code between adjacent captured images are completely the same coordinates after coordinate conversion. The reasons for this are, for example, that the photographing range may shift due to vibrations when the truck 31 is running, and that the detection position and size of the JAN code may change depending on how the JAN code appears in the photographed image. Things are mentioned.

The JAN code 241 with the read value "JAN001" appears in the Nth shot image P61 shown in FIG. Also, the JAN code 242 of the read value "JAN001" appears in the (N+1)th photographed image P62. These JAN codes 241 and 242 are assumed to be the same JAN code displayed on a cardboard box. However, when the coordinates of the JAN codes 241 and 242 are converted to the coordinates of the coordinate system of the synthesized image P63, the converted coordinates of the JAN codes 241 and 242 in the synthesized image P63 do not completely match.

Therefore, the result output processing unit 124, when the distance between the positions after the coordinate conversion of the JAN codes having the same read value between the adjacent photographed images is less than a predetermined determination threshold, determines that the JAN codes are the same JAN code. It is determined that Here, a method for determining the determination threshold will be described with reference to FIG. 15 below.

FIG. 15 is a diagram for explaining a determination threshold determination method for JAN code identification. This determination threshold may be determined, for example, based on the distance between the JAN codes when the cardboard boxes are actually closest to each other when the cardboard boxes are stacked.

In FIG. 15, the cardboard boxes 251 to 254 are stacked such that the lower left edge of the cardboard box 251, the lower right edge of the cardboard box 252, the upper left edge of the cardboard box 253, and the upper right edge of the cardboard box 254 are aligned. is exemplified. Labels 261 to 264 printed with JAN codes are attached to the lower left end of the cardboard box 251, the lower right end of the cardboard box 252, the upper left end of the cardboard box 253, and the upper right end of the cardboard box 254, respectively.

In such a case, it is considered that the distance between the JAN codes of labels 261-264 is the shortest. In the example of FIG. 15, it is assumed that the horizontal distance between JAN codes is 90 mm and the vertical distance is 70 mm. In this case, the minimum value of 70 mm is preset as the determination threshold.

Next, processing of the PC 100 will be described using a flowchart.
FIG. 16 is an example of a flowchart showing the overall processing of the PC.
[Step S11] A trolley loaded with cardboard boxes is pushed by the worker, and the trolley starts to move. Images are taken continuously by each camera as the cart moves across the cameras. The image receiving unit 121 of the PC 100 receives captured images from each camera and stores them in the image information 113 .

[Step S<b>12 ] The JAN code reader 122 detects the JAN code from each captured image stored in the image information 113 . The JAN code reading unit 122 identifies the position (coordinates) of the detected JAN code in the captured image and the vertical size of the JAN code in the captured image, and reads information indicated by the JAN code. The JAN code reading unit 122 registers the detected JAN code position, vertical size, and reading value in the JAN code reading information 114 in association with identification information indicating the detected captured image.

[Step S13] The image synthesizing unit 123 executes a process of calculating a shift width for each pair of photographed images that are consecutive in the horizontal direction.
[Step S14] The image synthesizing unit 123 performs a process of calculating an overlapping width for each pair of shot images that are consecutive in the vertical direction.

[Step S15] Using the calculated shift width and overlap width, the image synthesizing unit 123 converts the coordinates of each pixel of each captured image into the coordinate system of the panorama image in accordance with the above-described equations (12) and (13). Convert to coordinates.

[Step S16] The image synthesizing unit 123 creates a panorama image by synthesizing the captured images based on the coordinate conversion result. At this time, the pixel values of overlapping areas between the photographed images after the coordinate transformation are synthesized.

[Step S17] The result output processing unit 124 identifies the same JAN code between adjacent captured images, and eliminates duplication of JAN codes. Specifically, based on the coordinates after conversion, the result output processing unit 124 determines that adjacent captured images have the same read value and that the distance between the positions in the coordinate system of the panoramic image is less than a predetermined determination threshold. is determined to be the same JAN code. The result output processing unit 124 deletes one of the identical JAN codes from the JAN code reading information 114 . Between the photographed images adjacent in the horizontal direction, for example, the JAN code in the temporally front photographed image is deleted. Between vertically adjacent captured images, for example, the JAN code in the lower captured image is deleted.

[Step S18] Based on the JAN code on the panorama image (that is, the JAN code remaining in the JAN code reading information 114), the result output processing unit 124 calculates the read value of the JAN code and the number of JAN codes having the same read value. , and the created panorama image as the final result.

Note that if step S16 is not executed and no panorama image is output as the final result in step S18, then in step S15 only the coordinates of the JAN code detected from each captured image should be converted into the coordinate system of the panorama image. .

FIG. 17 is an example of a flowchart showing the procedure of shift width calculation processing. The processing of FIG. 17 corresponds to the processing of step S13 of FIG.
[Step S21] A horizontal processing loop up to step S28 is executed for each camera.

[Step S22] The horizontal pair processing loop up to step S27 is executed for each continuous photographed image pair (horizontally adjacent photographed image pair) among the photographed images photographed by the same camera.

[Step S23] The image synthesizing unit 123 identifies JAN codes having the same read value between the pair of captured images. Here, when a plurality of JAN codes having the same read value exist in one captured image, the image synthesizing unit 123 selects the JAN code having the largest X coordinate among them.

[Step S24] The image synthesizing unit 123 determines whether or not the corresponding JAN code was specified in step S23. If the corresponding JAN code can be specified, the process proceeds to step S25, and if the corresponding JAN code cannot be specified, the process proceeds to step S26.

[Step S25] The image synthesizing unit 123 calculates the shift width for synthesizing the captured image pair based on the difference between the X coordinates of the identified JAN codes.
[Step S26] The image synthesizing unit 123 uses the shift width calculated at the most recent execution of step S25 as the shift width when synthesizing the captured image pair to be processed.

[Step S27] When the processing of steps S23 to S26 has been performed for all pairs of captured images, the horizontal pair processing loop ends.
[Step S28] When the processing of steps S22 to S27 has been executed for all cameras, the horizontal processing loop ends.

FIG. 18 is an example of a flowchart showing the procedure of overlap width calculation processing. The processing of FIG. 18 corresponds to the processing of step S14 of FIG.
[Step S31] The image synthesizing unit 123 acquires the vertical sizes of all JAN codes detected from each captured image from the JAN code reading information 114, and calculates a histogram of the vertical sizes. The image synthesizing unit 123 determines an exclusion range for exclusion from the calculation of the average value of the vertical size based on the histogram. For example, a range of vertical sizes separated by a certain value or more from the vertical size at which the frequency peaks is determined as the excluded range.

[Step S32] A vertical processing loop up to step S40 is executed for each group of captured images captured by each camera at the same time.
[Step S33] A vertical image loop up to step S35 is executed for each photographed image included in the photographed image group.

[Step S34] The image synthesizing unit 123 determines the vertical size PQH of the JAN code detected from the captured image. If a plurality of JAN codes are detected from the captured image, the image synthesizing unit 123 excludes the JAN codes whose vertical size is included in the exclusion range determined in step S31 from these JAN codes, and removes the remaining JAN codes. The vertical size PQH is calculated by averaging the vertical sizes of the codes.

The image synthesizing unit 123 acquires the actual height QH of the JAN code from the JAN code information 112 and also acquires the vertical size IH of the captured image and the viewing angle θ of the camera from the camera setting information 111 . Using the calculated vertical size PQH, the obtained height QH, the vertical size IH, and the viewing angle θ, the image synthesizing unit 123 calculates the distance D from the camera to the object plane according to Equation (4).

[Step S35] When the process of step S34 is executed for all the captured images included in the captured image group, the vertical image loop ends.
[Step S36] The vertical pair processing loop up to step S39 is executed for each photographed image pair that is vertically continuous in the photographed image group.

[Step S37] The image synthesizing unit 123 acquires from the camera setting information 111 the height of the camera that captured each captured image included in the captured image pair. The image synthesizing unit 123 uses the acquired height of the camera to calculate the positions of the upper end and the lower end of the photographing range for each photographed image included in the photographed image pair according to equations (5) and (6).

[Step S38] The image synthesizing unit 123 uses the calculation result of step S37 to calculate the overlapping width PW corresponding to the captured image pair according to equation (11).
[Step S39] When the processing of steps S37 and S38 has been performed for all pairs of captured images, the vertical pair processing loop ends.

[Step S40] When the processes of steps S33 to S39 are executed for all the photographed image groups, the vertical processing loop ends.
Note that the processing functions of the devices (for example, the image processing device 1 and the PC 100) shown in the above embodiments can be realized by a computer. In that case, a program describing the processing contents of the functions that each device should have is provided, and the above processing functions are realized on the computer by executing the program on the computer. A program describing the processing content can be recorded in a computer-readable recording medium. Computer-readable recording media include magnetic storage devices, optical disks, semiconductor memories, and the like. Magnetic storage devices include hard disk drives (HDD) and magnetic tapes. Optical discs include CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray Discs (BD, registered trademark), and the like.

When distributing a program, for example, portable recording media such as DVDs and CDs on which the program is recorded are sold. It is also possible to store the program in the storage device of the server computer and transfer the program from the server computer to another computer via the network.

A computer that executes a program stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. The computer then reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Also, the computer can execute processing according to the received program every time the program is transferred from a server computer connected via a network.

1 Image processing device 1a Processing unit 2 Camera 3 Cart 4a-4c, 4a1, 4a2 Barcode P0 Synthetic image P1, P2 Photographed image S1-S4 Step

Claims (8)

to the computer,
In a state in which a plurality of boxes each containing a product and having a bar code indicating information of the contained product placed on each surface are loaded on a cart, the cart passes across in front of the camera. Obtaining a plurality of captured images continuously captured by the camera during the period of time,
detecting the coordinates of the barcode and the reading value of the barcode from each of the plurality of captured images;
extracting the coordinates of the barcode having the same read value between adjacent first captured image pairs among the plurality of captured images as corresponding feature points between the first captured image pairs;
Coordinates of the barcode detected from each of the plurality of captured images in a coordinate system of a panoramic image obtained by synthesizing the plurality of captured images based on the extracted coordinates of the corresponding feature points; outputting the reading value and the number of the barcodes arranged on the surfaces of the plurality of boxes based on the reading value of the code;
An image processing program that executes processing. In the extraction of the corresponding feature points, when a plurality of the barcodes having the same read value exist in one captured image of the first captured image pair, the horizontal coordinates of the barcodes are maximum or minimum. as one of the corresponding feature points,
2. The image processing program according to claim 1. In outputting the reading value and the number of the barcode, the amount of shift in the horizontal direction of one of the first captured image pairs with respect to the other when combining the first captured image pair is set to each coordinate of the corresponding feature point. calculated based on
The panorama image is generated by synthesizing the plurality of captured images using the shift amount for each of the first captured image pairs.
In the calculation of the shift amount, if the barcode having the same read value cannot be specified between one pair of captured images of the first pair of captured images, the other bar code prior to the one pair of captured images is determined. diverting the shift amount calculated most recently for the first captured image pair as the shift amount for the one captured image pair;
3. The image processing program according to claim 1 or 2. A plurality of the cameras are installed in the vertical direction,
acquiring the plurality of captured images from each of the plurality of cameras;
The coordinate system of the panoramic image is generated by synthesizing the plurality of captured images obtained from each of the plurality of cameras,
4. The image processing program according to claim 1. In outputting the reading value and the number of the barcode, among the plurality of photographed images corresponding to each of the plurality of cameras, from among the photographed images photographed by the plurality of cameras at the same time, the adjacent extracting a second captured image pair corresponding to a camera, and calculating, for each second captured image pair, an overlap width of one of the second captured image pairs with respect to the other when synthesizing the second captured image pair;
The panoramic image is generated by synthesizing the captured image group using the overlapping width for each of the second captured image pairs,
The overlap width is the number of vertical pixels of the barcode detected in each of the second captured image pair, the preset actual size of the barcode in the vertical direction, and the second captured image pair. calculated based on the parameters of the camera taken respectively,
5. The image processing program according to claim 4. In calculating the overlap width,
The number of vertical pixels of the barcode detected in each of the second captured image pairs, the actual size of the barcode in the vertical direction, the number of vertical pixels of each of the second captured image pairs, and the camera's calculating the distance between each of the plurality of cameras that captured the second captured image pair and the surface to be captured, based on the viewing angle;
calculating a vertical imaging range of each of the camera pairs on the imaging target plane based on the height of each of the camera pairs and the distance calculated for each of the camera pairs;
calculating the overlap width based on the calculated height of the overlap region between the imaging ranges;
6. The image processing program according to claim 5. the computer
In a state in which a plurality of boxes each containing a product and having a bar code indicating information of the contained product placed on each surface are loaded on a cart, the cart passes across in front of the camera. Obtaining a plurality of captured images continuously captured by the camera during the period of time,
detecting the coordinates of the barcode and the reading value of the barcode from each of the plurality of captured images;
extracting the coordinates of the barcode having the same read value between adjacent first captured image pairs among the plurality of captured images as corresponding feature points between the first captured image pairs;
Coordinates of the barcode detected from each of the plurality of captured images in a coordinate system of a panoramic image obtained by synthesizing the plurality of captured images based on the extracted coordinates of the corresponding feature points; outputting the reading value and the number of the barcodes arranged on the surfaces of the plurality of boxes based on the reading value of the code;
An image processing method that causes processing to be performed. In a state in which a plurality of boxes each containing a product and having a bar code indicating information of the contained product placed on each surface are loaded on a cart, the cart passes across in front of the camera. Obtaining a plurality of captured images continuously captured by the camera during the period of time,
detecting the coordinates of the barcode and the reading value of the barcode from each of the plurality of captured images;
extracting the coordinates of the barcode having the same read value between adjacent first captured image pairs among the plurality of captured images as corresponding feature points between the first captured image pairs;
Coordinates of the barcode detected from each of the plurality of captured images in a coordinate system of a panoramic image obtained by synthesizing the plurality of captured images based on the extracted coordinates of the corresponding feature points; a processing unit that outputs the read values and the number of the barcodes arranged on the surfaces of the plurality of boxes based on the read values of the codes;
An image processing device having

Images