KR102112721B1 - Image Processing System and Method of moving object - Google Patents

Abstract

Disclosed are an image processing system of a moving object which recognizes identification labels attached to or printed on the moving object to recognize or classify the moving object, and a method thereof. The image processing system of a moving object comprises: a lens; an image sensor in which a focal plane, through which an image of a photographed object passes through the lens, is inclined, not perpendicular to a central axis of the lens; and an image obtaining unit obtaining the image formed while passing through the focal plane of the inclined image sensor multiple times at a predetermined time difference as the photographed object moves. Accordingly, a structure is simple, the image processing speed is fast, and installation and operation costs can be reduced since a separate driving device for adjusting focus is not required.

Description

Image processing system and method of moving object}

The present invention relates to an image processing system and method of a moving object for recognizing or classifying moving objects in order to recognize identification labels attached to or printed on the moving object.

The image acquiring device for acquiring an image generally includes a lens and an image sensor, and the position of the focal point of the image varies depending on the position of the object to be photographed, so the position of the image sensor must also be moved accordingly.

FIG. 1 is a diagram for explaining that an image in focus is obtained by moving a position of an image sensor according to a distance of an object from a lens when photographing a general image.

The range of distances in which the image acquisition apparatus can shoot is determined according to its specification. Let's assume that the closest position that can be taken in focus is the first position 10 and the farthest position is the second position 20. If the object 101 is in the first position 10, the image 102 is formed at the position 120 corresponding to the first position 10 in accordance with the refractive principle of the optical system including the lens 110. The sensor should be located at 120, and when the object 105 is in the second position 20, the image 106 is formed at the position 140 corresponding to the second position 20, so that the image sensor is (140). ).

Therefore, the position of the focal point of the image varies according to the shooting distance range determined according to the first position 10 and the second position 120, and in order to obtain a focused image, the image sensor also has a moving range corresponding to the focal length. Since it must move properly and quickly within (between 120 and 140), a separate driving device is required to move the image sensor, and the structure becomes complicated.

According to one embodiment, a moving object image processing system and method are proposed that can quickly and simply obtain a focused image at multiple locations without a driving device for focus adjustment.

A moving object image processing system according to an embodiment includes a lens; An image sensor in which the focal plane through which the image of the photographed object passes through the lens is inclined, not perpendicular to the central axis of the lens; And an image acquisition unit that acquires an image formed while passing the focal plane of the tilted image sensor multiple times at a predetermined time difference as the photographing object moves.

The lens is disposed parallel to the article loading surface or the moving direction of the transport device carrying the photographed object, and the image sensor is disposed to be parallel to the article loading surface or the moving direction of the transport device carrying the photographed object. , It may be inclined at a predetermined angle to either side based on the moving direction.

The image sensor, which changes according to the size of the photographed object, corresponds to a point corresponding to a first focal length corresponding to a minimum distance from the lens to the photographed object, and a maximum distance from the lens to the photographed object Points corresponding to the second focal length may be inclined to be located at both ends of the image sensor.

The image acquiring unit may continuously photograph the photographed object at a predetermined time interval from entering into the angle of view of the lens as it moves and exiting.

In addition, the image acquisition unit may select one or multiple images in focus from among the multiple images continuously photographed, and recognize an identification mark included in the corresponding image.

In addition, when a focused image is selected from among the multiple images continuously photographed, the image acquisition unit may select data with a predetermined width of at least one line in units of pixel lines from the selected image to recognize an identification mark included in the corresponding image. have.

Further, the at least one of a distance, a position, and a volume to the photographing object may further include a measuring unit measuring a distance to the photographing object, and the image acquisition unit may use the measured distance information to determine the selected image. In addition, a focus band selector for selecting a position of a pixel line in focus may be further included.

The identification label may include any one of a barcode, QR code, text, and figure.

The moving object image processing method according to an embodiment of the present invention includes an image sensor in which a focal plane formed by an image of an object to be passed through a lens is inclined without being perpendicular to a central axis of the lens, and the inclined object is moved as the object is moved An image acquiring step of continuously acquiring an image formed while passing through the focal plane of the image sensor at a predetermined time difference; And selecting one or multiple images in focus from among the multiple images continuously acquired and recognizing identification marks included in the corresponding images.

In the acquiring step, as the photographing object moves, it may be continuously photographed at a predetermined time interval from entering and exiting the angle of view of the lens.

The step of recognizing the identification mark may recognize any one of a barcode, a QR code, a text, and a figure included in a plurality of images continuously photographed.

The method may further include measuring a distance to an object to be photographed, and recognizing an identification mark may include one or a plurality of images that are in focus among a plurality of images continuously acquired using the measured distance information. By selecting, the identification mark included in the corresponding image can be recognized.

Since the moving object image processing system and method according to an embodiment does not require a separate driving device for focus adjustment, the structure is simple, the image processing speed is fast, and installation and operation costs can be reduced.

1 is a view for explaining that an image in focus is obtained by moving a position of an image sensor according to a distance of an object from a lens during general image shooting
FIG. 2A is a view illustrating a configuration in which an image sensor is tilted without being parallel to a lens according to an embodiment of the present invention;
Figure 2b is a view for explaining the configuration and the principle of the mobile image processing system according to an embodiment of the present invention,
3 is a view showing a first implementation example of a moving object image processing system according to an embodiment of the present invention,
4 is a view showing a second implementation example of a moving object image processing system according to an embodiment of the present invention;
5 is a view for explaining obtaining a plurality of images in the image acquisition unit 230,
6 is a view showing a moving object image processing system according to another embodiment of the present invention,
7 is a flowchart of a moving object image processing method according to an embodiment of the present invention,
8 is a flowchart of a moving object image processing method according to another embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them will be clarified with reference to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the embodiments allow the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention pertains. It is provided to fully inform the holder of the scope of the invention, and the invention is only defined by the scope of the claims. The same reference numerals refer to the same components throughout the specification.

In the description of the embodiments of the present invention, when it is determined that a detailed description of known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted, and terms to be described later in the embodiments of the present invention These terms are defined in consideration of the function of the user, and may vary depending on the user's or operator's intention or custom. Therefore, the definition should be made based on the contents throughout this specification.

Combinations of each block in the accompanying block diagrams and steps in the flow charts may be performed by computer program instructions (execution engines), these computer program instructions being incorporated into a processor of a general purpose computer, special purpose computer, or other programmable data processing device. Since it can be mounted, the instructions executed through a processor of a computer or other programmable data processing device create a means to perform the functions described in each block of the block diagram or in each step of the flowchart.

These computer program instructions can also be stored in computer readable or computer readable memory that can be oriented to a computer or other programmable data processing device to implement a function in a particular way, so that computer readable or computer readable memory The instructions stored in it are also possible to produce an article of manufacture containing instructions means for performing the functions described in each block of the block diagram or in each step of the flowchart.

And since computer program instructions may be mounted on a computer or other programmable data processing device, a series of operational steps are performed on the computer or other programmable data processing device to create a process that is executed by the computer to generate a computer or other programmable It is also possible for instructions to perform the data processing apparatus to provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.

In addition, each block or each step can represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical functions, and in some alternative embodiments referred to in blocks or steps It should be noted that it is also possible for functions to occur out of sequence. For example, two blocks or steps shown in succession may in fact be performed substantially simultaneously, and it is also possible that the blocks or steps are performed in the reverse order of the corresponding function as necessary.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the embodiments of the present invention exemplified below may be modified in various other forms, and the scope of the present invention is not limited to the embodiments described below. Embodiments of the present invention are provided to more fully describe the present invention to those of ordinary skill in the art.

FIG. 2A is a view illustrating a configuration in which an image sensor is inclined and not parallel to a lens according to an embodiment of the present invention.

As described above, the image sensor 220 is generally disposed parallel to the lens 210, that is, perpendicular to the central axis 240 of the lens. However, in an embodiment of the present invention, it can be seen that the image sensor 220 is inclined as shown in FIG. 2A. The principle of obtaining a focused image through the tilted image sensor 220 will be described in detail through the embodiments of FIGS. 2B and 3 and 4.

2B is a diagram for explaining a configuration diagram and a principle of a moving object image processing system according to an embodiment of the present invention.

The moving object image processing system according to an embodiment of the present invention includes a lens 210, an image sensor 220, and an image acquisition unit 230.

The lens 210 may be implemented as a single lens, or a combination of various types of lenses, and light emitted from an object passes through the lens 210 and is condensed to the image sensor 220. Therefore, because the focal length is different, the position where the focused image is formed is also different.

The image sensor 220 forms an image of an object that has passed through the lens 210. At this time, the focal plane formed is inclined without being perpendicular to the central axis of the lens. In more detail, the lens 210 is disposed in parallel to the article loading surface or the moving direction of the transportation device carrying the moving object, but the image sensor 220 includes the article loading surface of the transportation device carrying the moving object. Alternatively, after being arranged parallel to the moving direction, it is inclined at a predetermined angle to either side based on the moving direction.

For a specific example of the degree of inclination, a point 270 corresponding to a first focal length corresponding to a minimum distance from the lens 210 to the object to be photographed, which changes according to the size or movement position of the object to be photographed, and the lens Points 260 corresponding to the second focal length corresponding to the maximum distance from 210 to the photographing object may be inclined to be located at both ends of the image sensor 210. Meanwhile, the image sensor 220 may be, for example, a two-dimensional image sensor.

A detailed tilt angle according to an example of a minimum distance and a maximum distance to a photographing object, a focal length of a lens, and an length of an image sensor is numerically described as follows.

First, the calculation of the focal length is as shown in Equation 1 below.

[Equation 1]

1 / a + 1 / b = 1 / f

Here, a is the distance between the subject and the lens, b is the distance between the lens and the image, and f is the focal length of the lens. Summarizing Equation 1, the distance b between the lens and the image is calculated as in Equation 2 below.

[Equation 2]

b = af / (a-f)

For example, assuming a case where the focal length of the lens is 35 mm and the distance to the object to be photographed varies within a range from 2,400 m to 1,600 mm, the distance of the image sensor is 35.78275 when the object to be photographed is located at a minimum distance of 1,600 mm. mm, the distance of the image sensor is calculated as 35.51797mm when the object is located at a maximum distance of 2,400mm, and the difference is 0.26478mm. Therefore, assuming that the length of one side of the image sensor is 10 mm, even if the distance difference between both ends is 0.26478 mm, both the minimum distance and the maximum distance of the object to be photographed can be covered.

If this is calculated as the tilted angle value, atan (0.26478 / 10) = 1.518 degrees, so the degree of inclination of the image sensor is only about 1.5 degrees. It is very effective because it can cover all distances up to 800mm.

The image acquisition unit 230 acquires the image formed while passing the focal plane of the image sensor 220 inclined as the photographing object moves, a plurality of times with a predetermined time difference. As an example, the image acquisition unit 230 may continuously photograph at a predetermined time interval from entering the angle of view of the lens 210 as it moves out of the lens 210 as it moves. The example of the time interval may vary depending on the moving speed of the object to be photographed, and if the moving speed is fast, the moving speed may be taken at a short interval, and if the moving speed is slow, the shooting speed may be taken at a long interval. Since it must be recognized, it can be set to shoot at least two times of any sized object.

On the other hand, the range of recognizing the shooting object is determined according to the characteristics of the lens 210, the characteristics of the image sensor 220 and the image acquisition unit 230, and the system of the present invention transfers the limited range of the shooting object (object). It should be set as an area, and is configured to move the photographing object (object) to the conveying area through a continuous conveying means such as a conveyor belt or a single-use conveying means such as AGV.

In addition, the image acquisition unit 230 may select a single focused image from a plurality of images continuously photographed and recognize an identification mark included in the corresponding image. The identification mark may be any one of barcode, QR code, text, and figure. For example, the recognition algorithms of other identification marks, such as barcode recognition, OR code recognition, character recognition, and figure, use common recognition algorithms such as barcode recognition and character recognition. Can be used.

3 is a diagram illustrating a first implementation example of a moving object image processing system according to an embodiment of the present invention.

The moving object image processing system according to an embodiment may be installed on the side of the transfer device 300 for moving the article.

The article 30, 32, and 34 are carried on the transfer device 300 to move, and the moving object image processing system is installed on the side of the transfer device 300 to photograph the side of the moving product. The angle of view of the lens 210 as the objects 30, 32, and 34 pass between the minimum shooting distance location 310 and the maximum shooting distance location 320 that the moving object image processing system according to an embodiment of the present invention can shoot The filming starts from the inside. For example, when the product 1 30 is continuously photographed, not all the images photographed due to the size of the product 1 are in focus, but when the first position 330 is reached, focus is achieved. On the other hand, in case of article 2 (32), when the second position 340 is reached, a focused image is obtained, and in the case of article 3 (34), when the third position 350 is reached, a focused image is obtained. You get.

That is, the position of the image on the inclined image sensor 220 is different because the focus position is different depending on the position of the article on the transport device 300 and the size of the article, but an image in focus can be obtained at any one location. , The image acquisition unit 230 may obtain an image in focus by continuously acquiring the image. Barcodes, QR codes, texts, figures, etc. are printed on the sides of the photographed articles 30, 32, and 34, and information about the article can be obtained through recognition of the identification mark.

4 is a diagram illustrating a second implementation example of a moving object image processing system according to an embodiment of the present invention.

The moving object image processing system according to an embodiment may be installed on the upper surface of the transport device 400 for moving the goods.

Goods 40, 42, 44, and 46 are moved on the transport device 400, and the moving object image processing system is installed on the upper surface of the transport device 400 to photograph the upper surface of the moving goods. As the moving object image processing system according to an embodiment of the present invention, the lens 210 is passed through the article 40, 42, 44, 46 between the minimum shooting distance 410 and the maximum shooting distance 420 The filming starts from entering the angle of view. For example, if the product 1 (40) is continuously photographed, not all the images photographed due to the size of the product 1 are in focus, but when the first position 430 is reached, focus is achieved. On the other hand, in the case of article 2 (42), when the second position 440 is reached, an image in focus is obtained, and in the case of article 4 (46), an image in focus is obtained when it reaches the third location (450). You get.

That is, the position of the image on the inclined image sensor 220 is different because the focus position is different depending on the position of the article on the transport device 400 and the size of the article, but an image in focus can be obtained at any one location. , The image acquisition unit 230 may obtain an image in focus by continuously acquiring the image. Barcodes, QR codes, texts, figures, etc. are printed on the sides of the photographed articles 30, 32, and 34, and information about the article can be obtained through recognition of the identification mark.

On the other hand, the transfer device (300, 400) may be a conveyor belt, for example, but not in the form of a belt, but a variety of transport devices that continuously move or move items in one direction by fixing or loading items such as rings and chains, and independent autonomy such as AGV. It may include all of the transport moving device. And it may be installed on both sides by combining the side installation of FIG. 3 and the upper installation of FIG. 4.

5 is a view for explaining obtaining a plurality of images in the image acquisition unit 230.

When the article 50 is carried on the transport apparatus 500, a plurality of images are acquired while moving, and a plurality of focused images can be obtained (510, 520). Since it is not possible to know where the identification mark is attached or printed on the article 50, it is possible to recognize the identification mark such as a barcode by obtaining a plurality of images at a focused location according to one article.

6 is a view showing a moving object image processing system according to another embodiment of the present invention.

The moving object image processing system may further include a measurement unit 610. The measuring unit 610 measures the distance to the product 1 (40). The distance to the article can be simply measured using ultrasonic waves, infrared sensors, or the like, and the position, volume, etc. can be measured using a three-dimensional sensor. That is, the distance to the article, which may vary depending on the size of the article, may be measured more accurately.

In addition, the image acquisition unit 230 may automatically select a position in focus from the image sensor 220 using the obtained distance information. That is, when the distance to the article is known, the distance to the focal point of the image can be known by Equations 1 and 2 described above, so that it is possible to know at which position the image is focused in the image sensor 220. In addition, when a focusing image is selected among a plurality of images continuously photographed, data having a predetermined width of at least one line in a pixel line unit may be selected from the selected image to recognize an identification mark included in the corresponding image. To this end, a focus band selector (not shown) may be further included, and the focus band selector may select a location of a pixel line in focus in the selected image using the measured distance information. Meanwhile, the band selection width of the pixel line may vary according to the size of the identification mark, which may be determined in response to the minimum size of the identification mark. In addition, the identification mark may be recognized using only one band image or the identification mark may be recognized by combining a plurality of consecutive band images.

7 is a flowchart of a moving object image processing method according to an embodiment of the present invention.

First, in the image acquiring step 710, the focal plane of the image sensor inclined as the photographed object moves as the focal plane through which the image of the photographed object passes through the lens is arranged to be tilted rather than perpendicular to the central axis of the lens The images concluded while passing are acquired continuously with a predetermined time difference. At this time, as the moving object moves, it continuously shoots at a predetermined time interval from entering into the angle of view of the lens until it leaves, and examples of the method and the time interval are as described above with reference to the aforementioned moving object image processing system. .

In addition, the recognition step 720 recognizes an identification mark included in the corresponding image by selecting a single focused image from a plurality of images acquired in this manner. For example, it is possible to recognize any one of barcodes, QR codes, texts, and figures included in a plurality of images photographed continuously.

8 is a flowchart of a moving object image processing method according to another embodiment of the present invention.

First, a distance to a photographing object is measured (810). The method of measuring the distance is as described above with reference to FIG. 6. In addition, the image formed while passing through the focal plane of the inclined image sensor as a moving object is moved from an image sensor in which the focal plane through which the image of the object to be shot passes through the lens is disposed is not perpendicular to the central axis of the lens. Obtained continuously over time (820). At this time, as the moving object moves, it continuously shoots at a predetermined time interval from entering into the angle of view of the lens until it leaves, and examples of the method and the time interval are as described above with reference to the aforementioned moving object image processing system. .

Next, by using the distance information measured in all of the multiple images to determine the position in focus within the image to extract the area (830). That is, since the positions in focus are located in the same pixel (line) for cargoes of the same height, a band image of pixels in the corresponding region is used. Then, the identification mark included in the corresponding image is recognized (840).

Meanwhile, when a focused image is selected among a plurality of images continuously photographed, data having a predetermined width of at least one line in a pixel line unit may be selected from the selected image to recognize an identification mark included in the corresponding image. Here, the band selection width of the pixel line may vary according to the size of the identification mark, which may be determined in correspondence with the minimum size of the identification mark. That is, the measured distance information can be used to select the position of the pixel line in focus in the selected image.

Meanwhile, in the embodiment of the present invention, the image processing system for moving object recognition is mainly described, but it may be applied to an image inspection device, an acquisition device, and an imaging device for various purposes, and the use is not limited to the embodiment of the present invention.

Therefore, the moving object can be applied not only to logistics, but also to various uses for capturing and recognizing moving objects.

So far, the present invention has been focused on the embodiments. Those skilled in the art to which the present invention pertains will understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in terms of explanation, not limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent range should be interpreted as being included in the present invention.

Claims (13)

lens;
The focal plane through which the image of the object to be shot passes through the lens is arranged to be inclined without being perpendicular to the central axis of the lens, but the degree of inclination depends on the focal length of the lens, the minimum distance to the object, and the maximum distance An image sensor in which a focal length varies according to each point in the image sensor without adjusting the distance of the optical path by calculating and setting in advance; And
As the photographing object moves, the image formed while passing through the focal plane of the tilted image sensor is continuously photographed multiple times at predetermined time intervals from entering and exiting the angle of view of the lens, each point within the image sensor Moving image processing system, characterized in that it comprises an image acquisition unit for acquiring one or a plurality of different focus images according to. According to claim 1,
The lens is disposed parallel to the moving surface or the article loading surface of the transport device for moving the photographing object,
The image sensor is arranged so as to be parallel to an article loading surface or a moving direction of the transport device carrying the moving object, and then a moving object image processing system characterized in that it is inclined at a predetermined angle to either side based on the moving direction. . The method of claim 1, wherein the image sensor
A point corresponding to a first focal length corresponding to a minimum distance from the lens to the object to be photographed, which varies according to the size of the object to be photographed, and a second focal length corresponding to a maximum distance from the lens to the object to be photographed Moving object image processing system, characterized in that the points corresponding to the inclined arrangement to be located at both ends of the image sensor. The method of claim 1, wherein the image sensor
A moving object image processing system, characterized in that it is a two-dimensional image sensor. delete The method of claim 1, wherein the image acquisition unit
A moving object image processing system, characterized in that one or more images in focus are selected from among a plurality of images continuously photographed to recognize identification marks included in the corresponding images. The method of claim 1, wherein the image acquisition unit
A moving object characterized in that when a focused image is selected among the multiple images continuously photographed, a data having a predetermined width of at least one line in a pixel line unit is selected from the selected image to recognize an identification mark included in the corresponding image. Image processing system. The method of claim 7,
Further comprising a measuring unit for measuring the distance to the object to be detected by detecting at least one of the distance, position, volume to the object to be photographed,
The image acquisition unit further comprises a focus band selection unit for selecting a position of a pixel line in focus from the selected image using the measured distance information. The method of claim 6,
The identification mark is a mobile image processing system, characterized in that it comprises any one of a barcode, QR code, text, graphics. The focal plane through which the image of the object to be photographed passes through the lens is arranged to be inclined without being perpendicular to the central axis of the lens, but the degree of inclination is determined in advance according to the focal length of the lens and the minimum distance to the object to be photographed. By calculating and setting, from the image sensor whose focal length varies depending on each point in the image sensor without adjusting the distance of the optical path, the image formed while passing through the focal plane of the tilted image sensor as the object to be moved moves, An image acquiring step of acquiring one or a plurality of different focus images according to each point in the image sensor by photographing multiple times in succession at predetermined time intervals from entering the view angle to exiting; And
And a recognition step of selecting one or a plurality of images in focus from among the plurality of images photographed in succession and recognizing an identification mark included in the corresponding images. delete 11. The method of claim 10, Recognizing the identification mark is
A moving object image processing method characterized by recognizing any one of a barcode, a QR code, a text, and a figure included in a plurality of images continuously photographed. The method of claim 10,
Further comprising the step of measuring the distance to the subject,
The step of recognizing the identification mark is characterized in that, by using the measured distance information, one or more images in focus are selected from among the plurality of images acquired in succession to recognize the identification mark included in the corresponding image. Moving object image processing method.

Images