WO2016022085A2

WO2016022085A2 - An efficient method and system for rapidly and correctly counting thin stacked objects

Info

Publication number: WO2016022085A2
Application number: PCT/TR2015/000294
Authority: WO
Inventors: Halil AYDIN; Oguzhan URHAN; Ahmet Burak GOKBAYRAK
Original assignee: Bss Yazilim Muhendislik Ve Danismanlik Hizmetleri San. Ve Tic. Ltd. Şti.
Priority date: 2014-08-06
Filing date: 2015-08-05
Publication date: 2016-02-11
Also published as: WO2016022085A3; TR201409167A2

Abstract

The invention relates to coinciding a linear image sensor positioned parallel to objects to be counted, with the one-dimensional signal obtained by calculating the projection of the image formed with the sensor's vertical movement onto the horizontal axis, and to developing an efficient system and method based on the counting of objects after filtering possible ambient noises with signal processing techniques.

Description

DESCRIPTION

AN EFFICIENT METHOD AND SYSTEM FOR RAPIDLY AND CORRECTLY COUNTING THIN STACKED OBJECTS Subject of the Invention

This invention relates to a method and system developed for counting layers of sheets. The invention particularly relates to a method and system which rapidly and correctly detects numbers of stacked materials such as paper, carton, cardboard or sheet, etc. obtained using a linear camera and motion assembly by performing various filtering processes and comparison tests after converting raw data into a unidimensional signal.

Present Technical Field Many production facilities, primarily printing houses, paper mills, those manufacturing sheets relating to plastic and wood attach the number of products which they send onto the product with a label during the shipment of the sheet shaped products to the client. In the past, the counting of products was mainly done manually or visually using an operator (manpower) in both delivering shipments and also receiving goods. Applications, in which this counting method is used are still present. However, due to problems such as operators not working at the same performance and getting tired in time, counting errors arise. Furthermore, employment costs and loss of time remain as separate issues. Therefore, systems and methods were developed for counting said products with mechanic, electronic and computer-aided systems.

It is important to correctly determine the number of products in the warehouse or products to be dispatched to the customer right before the shipment, with sheet products such as paper, carton and cardboard. The products are packaged by being stacked before the shipment process. The counting process should be performed after the products have been stacked and right before the packaging process. For counting these types of materials, the method of dividing the height of the product to be counted with the height of the unit product after measuring the height of the product to be counted manually or with electronic systems may be employed if the unit product measurement is known. Such a system is disclosed in the U.S. Patent Document No. 4,417,351. However, it is not possible to determine an invariable and correct height for the materials due to both the properties of the materials used in the production process and the tolerances in the production process. For instance, counting flexible materials such as paper and plastic with this method has been found to cause substantial incorrect counts. Due to similar reasons, it is also impossible to correctly determine the number of materials by weighing these types of materials.

In the known state of the art, the counting of sheets is disclosed in U.S. Patent No. 5,040,196. In Patent No. US5040196, the counting process is performed with an approach which calculates correlations on a one-dimensional signal obtained with a linear photocell array moving with the aid of an operator. However, this method requires the aid of an operator, and also requires additional calibration for different types of materials and cannot correctly count the materials which are stacked irregularly and/or with spaces. In the known state of the art, U.S. Patent No 5,534,690 discloses a method for counting stacked banknotes. This method is for detecting the number of banknotes using techniques such as negative imaging, differentiation, binarization, filtering, etc. on an image created with the movement of a two-dimensional CCD or a linear CCD. However, the method and system are created according to the image sensors being positioned quite close to the objects to be counted. Therefore, in cases where it is not possible for the distance between the materials to be counted in practical terms with the counting system to be very short, it is not possible for the system to operate efficiently. In the known state of the art, U.S. Patent No. 6,157,457 (or European Patent No. EP0861477 of the same patent) states that counting may be done more efficiently with the analysis of the taken image through the net area depth improved by taking an image with a rotating mirror mechanism by mentioning the mechanically high cost of moving the image sensor and the high cost and poor resolution of the CCD image sensor. It is also mentioned that beams are sent to the stacked sheets and that the signal obtained with the reflection of this beam is measured. It is furthermore mentioned that the beam source may be left constant for this process and a beam may be sent to all of the sheet surfaces with a rotating mirror and a reflection may be obtained. However, nowadays, sensors of relatively high resolution may be obtained at a suitable cost with advances in sensor technology. Consequently, it is considered that there is no need for such equipment or at least that the cost thereof may be high. Furthermore, the type of technique that the counting is performed with is not stated from the modulated signal (which is transmitted, then hits the material and returns) obtained with the system suggested in this patent document.

In the known state of the art, European Patent Publication No. EP0743616 A2 suggests a counting apparatus and method to prevent the processing load which the two-dimensional sensors bring, with the use of a linear sensor. The linear sensor is placed vertical to the objects to be counted. Thus, the light information regarding a vertical section of the objects to be counted was gathered with linear sensors, a band pass filter was then applied about the dominant frequency with the Fourier Transform and then, the candidate positions of the objects was determined with the use of the inverse Fourier Transform. Afterwards, areas which could not be objects was distinguished using the neighbouring relation between neighbouring pixels. If a distortion of periodicity in the analysed signal is determined, the intervals are filled with the interpolation method, thus it is assumed that there are also objects in these areas. It was observed that there may be two significant issues relating to this method. The first thereof is that the quality of the raw signal to be obtained will be low if materials having a height of for example a few metres are counted due to the location of the linear camera compared to the material to be counted, and the visual field limitation of the of the optic which may be employed. When the visual field of the optic is increased, it is thought to be impossible for this method to correctly count without taking additional measures due to optical distortions. On the other hand, using the assumption that there are no spaces between the materials to be counted, this method fills this area with interpolation if the periodicity in the one-dimensional signal is distorted. Whereas, spaces between materials is frequently encountered in practice. The assumption of this method is of a structure to cause overcounting, i.e. the desired yield to not be obtained.

U.S. Patent Nos. 6,091,792 and 6,137,855 (which are both the same patent), encountered in the known state of the art disclose a method and system for counting corrugated articles by determining the sinusoidal structure of the corrugated articles with image processing on images gathered by moving an area scanning camera in a vertical direction with a mechanical assembly. Issues due to more than one images being combined in this system, the system's calibration and a two-dimensional signal being directly processed, and work load are thought to prevent an efficient counting system.

The systems and methods mentioned above have disadvantages due to both the counting accuracy and the calculational load of the employed technique. The inadequacies and issues in the employed techniques for counting stacked materials without the aid of an operator rendered it necessary to make an improvement in the related technical field. Object and Description of the Invention

The present invention aims a development relating to a counting system and method for efficiently counting stacked materials in order to eliminate the drawbacks of the other systems and methods mentioned above and to provide new advantages to the related technical field.

The main object of the invention is to coincide a linear image sensor positioned parallel to objects to be counted, with the one-dimensional signal obtained by calculating the projection of the image formed with the sensor's vertical movement onto the horizontal axis, and to develop an efficient system and method based on the counting of objects after filtering possible ambient noises with signal processing techniques.

In order to fulfil all the objects mentioned above and arising from the following detailed description, the present invention relates to an object counting system and method comprising the following steps to be used in counting systems of stacked materials:

- receiving light returning from objects lighted with a moving or stationary light source with a linear sensor moving in the direction of stacked objects and converting this into an electrical signal in a quantitative form,

- rendering the two-dimensional signal of a separate and quantitative form, formed with the linear sensor being moved with a motor assembly, one-dimensional through horizontal projection calculation,

- filtering noises formed by various reasons in a one-dimensional signal and determining possible object points on a one-dimensional signal with the analysis of this signal, - determining the starting and end points of the stack through a one-dimensional signal.

Said counting system and method is characterised in comprising the following process steps:

- the linear sensor positioned parallel to the objects to be counted converting the two-dimensional signal obtained with the movement of the sensor towards the stack, into a one-dimensional signal by calculating the projection in the other direction,

- again, determining the starting and end points of the cartons (or other types of sheets) with a one-dimensional signal by determining the signal structure, which is rather different from the average value of the obtained one-dimensional signal,

- correctly counting the number of objects with a two-dimensional data analysis only on determined candidate object areas after filtering possible ambient noises in the one-dimensional signal,

- the counting results obtained as a result of the process performed only in a certain area (region of interest - ROI) being performed in more than one areas in the previous stage being evaluated with the majority voting approach.

The following figures will be benefited from for a better understanding of the system of the invention.

Description of the Drawings

Figure 1 is a schematic illustration showing the main components of the system of the invention.

Figure 2 is a graphic view of the exemplary one-dimensional signal formed with the image's projection in the opposite direction of the movement of the two-dimensional image obtained after the movement of the linear camera in the system of the invention. Figure 3 is an exemplary graphic view showing the filtered version to eliminate the disruptive signals of the signal in Figure 2 in the system of the invention.

Figure 4 is a drawing showing marked object positions determined on the signal in Figure 3 in the system of the invention. Figure 5 is an illustration showing marked objects obtained after performing a two- dimensional analysis and eliminating some points around the peak points obtained in Figure 4 in the system of the invention. Herein, each marked point corresponds to an object (carton or other layers).

Figure 6 is a flow chart showing how the system of the invention functions.

Reference Numbers of the Parts Helping the Invention To Be Described

1- Linear sensor 2- Light source

3- Optical system

4- Motion system

5- Control unit

6- Platform or palette 7- Objects to be counted

7a- Linear scanning area

8- Converter or digitiser

9- Screen or interface a- Upward movement

b- Downward movement

c- Peak points (within the graph)

d- Possible errors or possible faulty candidates (in the process of filtering the graph comprising the peak points)

x- Horizontal pixel position (for the graph axis)

y- Avergae luminosity value (for the graph axis)

Stage Codes of the Flow Chart Given in Figure 6

100- Begin counting

110- Bring the linear sensor to the initial position 120- Take image in the determined exposure time and add to the previous image 130- Move the linear sensor in determined amounts

140- Have all the objects to be counted been scanned?

150- Count objects with a hybrid approach on the taken image

160- Transmit the result to a screen or desired interface

170- End counting

Detailed Description of the Invention The invention relates to an efficient method and system for rapidly and correctly counting stacked objects. In the detailed description given below, the novelty of the invention is disclosed with non-limiting examples for the better understanding of the subject. Accordingly, in the description below and the figures, a system and method developed for counting stacked objects is disclosed.

Figure 1 schematically shows the components belonging to the counting system enabling the practicality of the method of the invention. Accordingly, in the stacked object counting system, the objects to be counted (7) on a platform (6) are illuminated with a light source (2). The main reason for this illumination is that the image signal to be gathered with a linear sensor (1) by passing over an optical system (3) is at a desired level. The linear sensor (1) is fundamentally an image capturing device. In principle, the linear sensor (1) converts the light which hits the linear scanning area (7a) on the front surface of objects (7) whose exit from the light source (2) will be counted, returns and falls thereon into an electrical signal for a certain amount of time (exposure time). Typically, the amount of light falling onto the linear sensor (1) being high increases the amplitude of the electrical signal which this sensor will produce. This amplitude being sufficiently high is significant in terms of providing distinctiveness in the transitions of objects. As the linear sensor (1) to be employed may provide this electrical signal directly as a quantitative and discretely timed signal, a quantitative and discretely timed signal to be processed by a control unit (5) may also be obtained using a converter or digitiser (8). It is significant for the linear sensor (1) to take a linear image. In the known state of the art, in Patent Document Publication No. EP0743616A2 unlike the approach of vertical placement regarding the disclosed objects to be counted, it is positioned parallel to objects to be counted. In other words, to describe this in a more detailed manner, to show the width of the "n" linear sensor from the stack opposite it with the linear sensor's vertical movement, "1 x n" image slices are taken. Therefore, a row of images are taken in the stacking direction in the linear scanning area (7a) of the objects to be counted (7) opposite the system. In order to be able to count with this approach, the motion system (4) directed by the control unit (5) should move the objects to be counted (7) with the linear sensor (1) upwards (a) or downwards (b) in the stacking direction. The location where linear scanning is canied out, is the linear scanning area (7a) which is the front surfaces of the objects to be counted (7). This is not an area, but a line. The motion system (4) moves the linear sensor (1), optical system (3) and light source (2) in an upward direction of movement (a) or downward direction of movement (b). Herein, the direction of movement is essentially not very significant. What is significant is the speed of the movement and the duration of taking the image. The exposure time of the linear sensor (1) is not kept long due to the movement herein. Otherwise, a blurry image may be formed in the direction of movement. There is a significant relation between the speed of the motion system (4) moving the linear sensor (1) with the exposure time and the severity of the light source (2). It is possible for the invention to rather rapidly (1 m/s and higher) move the motion system (4) and thus the linear sensor (1) to be able to rapidly (e.g. in a few seconds) count stacks (objects to be counted (7)) which are a few metres long. The exposure time of the linear sensor (1) may need to be kept low for a blurriness issue to not occur in the image to be obtained. In this case, as the amount of light falling onto the linear sensor (1) may decrease, the light source (2) may need to provide highly intense light. If the light source (2) is highly intense, there may be light bursts in the image. Therefore, the exposure time of the linear sensor (1), the lens aperture of the optical system (3) and the intensity of the light source (2), and the motion speed of the linear sensors (1) moved with the motion system (4) should be balanced. These parameters may be theoretically calculated as well as being experimentally determined. The first location that the linear sensor (1) directly or indirectly transmits the images it has taken is the control unit (5) (with the converter or digitiser (8)). After the count carried out by the control unit (5), the count result may be transmitted to screens or interfaces (Θ).

The system of the invention and the method to be applied for this system to be carried out, is conducted according to the flow stated below. This method is demonstrated in the flow chart given in Figure 6. For this reason, the following stages are carried out. In the begin counting (100) stage, the objects to be counted (7) which could be carton, paper or another sheet were aligned on a platform or palette (6). Herein, the counting device is moved close to the linear scanning area (7a) which is the front face of the objects to be counted (7) by being certain that the objects to be counted (7) are straight and parallel to the ground.

With the bring the linear sensor to the initial position (110) stage, the light source (2) and the optical system (3) are brought to the initial position so as to face the lowermost part and the uppermost part of the linear scanning area (7a). It is not significant for the system to begin from the top or the bottom. What is significant is for the area to be scanned, to be seen clearly by the optical system (3) and for the movement to be able to begin towards the other direction.

In the take image in the determined exposure time and add to the previous image (120) stage, linear images are taken with the optical system's (3) light source (2) illuminating the linear scanning area (7a) well and with this transmitting the image to the linear sensor (1) correctly. The images taken afterwards are added to the previous images.

With the move the linear sensor (1) in determined amounts (130) stage, the upward movement (a) or downward movement (b) of the linear sensor (1) are enabled. Taking a linear image is carried out with the synchronised movement of the motion system (4) and the control unit (5) of the linear sensor (1).

In the have all the objects to be counted been scanned? (140) stage, whether all the images of the objects to be counted (7) have been obtained is controlled. If all the images have been obtained, the process is continued with the next stage; if they havent been obtained, the take image in the determined exposure time and add to the previous image ( 20) stage is returned to again and the process must be repeated. In the count objects with a hybrid approach on the taken image (150) stage, the obtained images are converted into quantitative numbers.

With the transmit the result to a screen or desired interface (160) stage, the obtained quantitative numbers are transmitted to a screen or an interface (9). With the end counting (170) stage, the counting process is terminated.

After the method carried out to execute the system of the invention is disclosed in this manner, the operation's details of the system of the invention and the components to be obtained as a result of the performed processes are described below with examples (graph examples).

An exemplary graph obtained as a result of the two-dimensional image obtained as a result of the movement of the linear sensor in the stacked direction (upward movement (a) or downward movement (b)) of the objects to be counted (7) being rendered one- dimensional with the projection calculation (e.g. average) in the opposite direction of the direction of movement, and selecting a certain area on this signal, and this signal being selected is present in Figure 2. In this graph, the pixel position horizontal to the "X" axis is termed as the average luminosity value to the "Y" axis. The peak points (c) in this graph are the number of candidates of the objects to be counted (7). Herein, number of candidates refers to the not yet confirmed number of objects to be counted (7). This process corresponds with the count objects with a hybrid approach on the taken image (150) stage. This stage is the stage containing the most significant know how of the system and method of the invention.

It is possible to determine the end point of the platform or palette (6), on which the objects to be counted (7) are found, i.e. the starting point of the objects to be counted (7) using the difference of the average amplitude level via this obtained one- dimensional signal (signal refers to the graph having X and Y axes) if the system continues to be described with said exemplary graph (Figure 2). This detection is carried out by determining the point where the objects to be counted (7) end in the image using the difference of the average amplitude level via this said one-dimensional signal (graph value). A one-dimensional signal free from disruptive effects is obtained by removing (filtering) predetermined suitable frequency components with methods such as the Fourier transform, wavelet transform, etc. according to the characteristic of the object to be counted (7) from the obtained one-dimensional signal (image). An exemplary graph result relating to this is present in Figure 3. As seen herein, the sudden transitions have been softened and the peak points (c) have been sharpened, unlike Figure 2. Thus, possible faulty detections have been prevented during the detection of peak points (c). It should not be forgotten that this operation is incorporated within the count objects with a hybrid approach on the taken image ( 50) stage. Thus, candidates of objects to be counted (7) are determined with the detection of peak points made via the filtered or refined signal. An exemplary graph result relating to this is present in Figure 4. As will also be seen in this graph, the determined peak points are the not yet confirmed number of the objects to be counted (7).

In this said stage (the count objects with a hybrid approach on the taken image (150) stage), the process of predetermining the number of objects by eliminating possible faulty candidates (d) after additional controls performed according to the structure of the object to be counted on the two-dimensional image around these candidate positions is present. An exemplary graph result relating to this is present in Figure 5. Possible errors (d) were prevented or filtered in this graph.

After evaluating the results obtained after repeating the aforementioned process steps on a one-dimensional signal for more than one areas, with a majority voting approach, the number of the objects was determined. The processes of displaying the number of determined objects on a screen, printing this or transmitting this to another system were carried out with the transmit the result to a screen or desired interface (160) stage.

In the method of the invention, it is possible to count by processing not only one- dimensional and two-dimensional signals, but primarily one-dimensional signals, then two-dimensional signals where a predetermination is made. Also, the row data is independently processed in the form of more than one pieces of a certain size, and the results thereof are evaluated through a majority voting. The rapid formation of an image comprising information with the optimum balance of a linear sensor (1), lens aperture in the optical system (3), exposure time and amount of lighting occurs with the control unit (5), converter and digitiser (8). Counting objects in the shape of a sheet to be counted with the application of the method and system of the invention is faster and more correct according to the known state of art. The method and system of the invention may be employed for the counting of all sheets such as paper, cardboard, plastic, metal, etc. The linear sensor (1), light source (2), optical system (3), motion system (4), control unit (5), preferably a converter or digitiser (8) and a screen or interface (9) which are the main components for applying said method will be able to be applied to any kind of machine design with a software enabling said components to be configured together in a desired flow.

Claims

1. A system and method for counting objects in the form of stacked sheets characterised in comprising the following stages:

- Begin counting (100)

- Bring the linear sensor to the initial position (110)

- Take image in the determined exposure time and add to the previous image (120)

- Move the linear sensor in determined amounts (130)

- Have all the objects to be counted been scanned? (140)

- Count objects with a hybrid approach on the taken image (150)

- Transmit the result to a screen or desired interface (160)

- End counting (170)

2. A system for counting objects in the form of stacked sheets characterised in comprising at least one linear sensor (1) detecting and displaying the linear scanning area (7a) on the front faces of objects to be counted on a platform (6); at least one light source (2); at least one optical system (3); a motion system (4) moving said linear sensor (1), light source (2) and optical system (3) upwards and downwards; a control unit (5) enabling said motion system (4) to operate in synchronisation with the linear sensor (1), light source (2) and optical system (3) and performing the counting process; preferably a converter (8) converting the analogue signal from the linear sensor (1) into said control unit (5) if necessary; and a screen or interface (9) providing information converted into a number from said control unit (5).

3. A method according to claim 1, characterised in comprising the stages of the objects to be counted (7) being on a platform (6), the counting device being parallel to the objects to be counted (7) and the objects to be counted (7) being moved closer to the linear scanning area (7a) which is the front face in the begin counting (100) stage.

4. A method according to claim 1 , characterised in comprising the stage of bringing the light source (2) and the optical system (3) to the initial position so as to face the lowermost part and the uppermost part of the linear scanning area (7a) in the bring the linear sensor to the initial position (110) stage.

5. A method according to claim 1 , characterised in comprising the stage of the optical system (3) and the light source (2) illuminating the linear scanning area

(7a) and transmitting the image towards a linear sensor (1) in the take image in the determined exposure time and add to the previous image (120) stage.

6. A method according to claim 1 , characterised in comprising the stage of the upward movement (a) or downward movement (b) of the linear sensor (1) being enabled, and taking a linear scanning image with the synchronised movement of the motion system (4) and the control unit (5) of the linear sensor (1) in the move the linear sensor in determined amounts (130) stage.

7. A method according to claim 1 , characterised in comprising the stage of controlling whether all the images of the objects to be counted (7) have been obtained, continuing to the next stage if all the images have been obtained, returning to the take image in the determined exposure time and add to the previous image (120) stage if they have not been obtained in the have all the objects to be counted been scanned? (140) stage.

8. A method according to claim 1 , characterised in comprising the stage of converting the taken images into quantitative numbers in the count objects with a hybrid approach on the taken image (150) stage.

9. A method according to claim 1 , characterised in comprising the stage of transmitting the obtained quantitative numbers to a screen or interface (9) in the transmit the result to a screen or desired interface (160) stage.

10. A method according to claim 1, characterised in comprising the stage of ending the counting process in the end counting (170) stage.

11. A method according to claim 1 , characterised in comprising the following stages in the count objects with a hybrid approach on the taken image (150) stage: rendering the two-dimensional image obtained as a result of the movement of the linear sensor in the stacked direction (upward movement (a) or downward movement (b)) of the objects to be counted (7), one-dimensional with the projection calculation in the opposite direction of the direction of movement or another method, and determining the number of candidates of the objects to be counted (7) by selecting a certain area on this signal,

determining the end point of the platform (6) on which the objects to be counted (7) are found using the difference of the average amplitude level via the obtained one-dimensional image (signal), obtaining a one- dimensional signal free from (filtered) disruptive effects by removing predetermined suitable frequency components with methods such as the Fourier transform, wavelet transform, etc. according to the characteristic of the object to be counted (7) from the obtained one-dimensional signal, determining candidates of objects to be counted (7) with the detection of peak points made via the filtered or refined signal or determining the confirmed numbers,

predetermining the number of objects by eliminating possible faulty (d) candidates after additional controls performed according to the structure of the object to be counted (7), on the two-dimensional image around the candidates and confirmed numbers of objects to be counted (7), determining the number of the objects after evaluating the results obtained after repeating these mentioned process steps on a one- dimensional signal for more than one areas, with a majority voting approach.

12. A method according to claim 1 , characterised in counting by processing not only one-dimensional and two-dimensional signals, but primarily one- dimensional signals, then two-dimensional signals where a predetermination is made in the method of the invention.

13. A method according to claim 1 , characterised in independently processing the row data in the form of more than one pieces of a certain size and evaluating the results thereof through a majority voting.

14. A method according to claim 2, characterised in comprising a control unit (5) and converter (8) forming a rapid image comprising information with the optimum balance of a linear sensor (1), lens aperture in the optical system (3), exposure time and amount of lighting.