WO2024087121A1

WO2024087121A1 - Method, electronic device and computer program product for generating a path

Info

Publication number: WO2024087121A1
Application number: PCT/CN2022/128070
Authority: WO
Inventors: Yan Xu; Cheng Peng; Dong-Jin Kim
Original assignee: Abb Schweiz Ag
Priority date: 2022-10-27
Filing date: 2022-10-27
Publication date: 2024-05-02

Abstract

A method, electronic device and computer program product for generating a path. The method comprises: receiving a point cloud, the point cloud indicating an outline of an object (202); determining a centroid of the point cloud (204); selecting a first point from the point cloud (206); and determining a second point adjacent to the first point based on a step angle (208). By applying the method, the path of the point cloud of the various shapes of the object can be determined, no matter how complicated the shape is.

Description

METHOD, ELECTRONIC DEVICE AND COMPUTER PROGRAM PRODUCT FOR GENERATING A PATH

FIELD

Example embodiments of the present disclosure generally relate to the field of machining industry, and more particularly, to a method, an electronic device and a computer program product for determining a path.

BACKGROUND

In the shoe-making process, glue spraying on shoe soles is a process that takes a lot of labor and time. Besides, glue spraying often has toxic effects on human body. Therefore, it is required to spray the glue on the shoe soles in an intelligence and automation manner. Currently, there are many kinds of shoe sole spraying methods on the market. However, since the shoe shape is non-standard, and has complex and diverse shapes, most of these approaches have drawbacks and thus still have room for improvements.

SUMMARY

In general, example embodiments of the present disclosure provide a method, an electronic device and a computer program product for determining a path.

In a first aspect, a method of determining a path is proposed. The method comprises: receiving a point cloud, the point cloud indicating an outline of an object; determining a centroid of the point cloud; selecting a first point from the point cloud; and determining a second point adjacent to the first point based on a step angle.

According to the example embodiments, the path of the point cloud of the various shapes of the object can be determined, no matter how complicated the shape is.

In some example embodiments, the step angle is constant.

In some example embodiments, the method further comprising: determining each distance between each point of the point cloud and the centroid; and determining a global maximum distance according to each distance.

In some example embodiments, the method further comprising: determining a predetermined angle; determining a local set of point from the point cloud based on the predetermined angle; and determining a local maximum distance between the local set of points and the centroid; wherein the step angle is determined according to the predetermined angle, the global maximum distance and the local maximum distance.

In some example embodiments, the method further comprising: determining a coordinate system in which the point cloud is positioned; determining an origin of the coordinate system; and translating the point cloud in the coordinate system to allow the centroid to coincide with the origin.

In some example embodiments, the method further comprising: denoising the point cloud before determining a centroid of the point cloud.

In some example embodiments, the method further comprising: splitting the point cloud from a background.

In some example embodiments, the method further comprising: projecting the point cloud onto a two-dimensional plane to form a projection; fitting the projection with a spline curve; calculating a slope of each points on the spline curve; and calculating an equidistant line based on the slope.

In a second aspect, an electronic device for determining a path is proposed. The device comprising: a receiving module configured to receive a point cloud, the point cloud indicating an outline of an object; a centroid determining module configured to determine a centroid of the point cloud; a selecting module configured to select a first point from the point cloud; and a point determining module configured to determine a second point adjacent to the first point based on a step angle.

In some example embodiments, the step angle is constant.

In some example embodiments, the device further comprising: a distance determining module configured to determine each distance between each point of the point cloud and the centroid; and a global maximum determining module configured to determine a global maximum distance according to each distance.

In some example embodiments, the device further comprising: an angle determining module configured to determine a predetermined angle; a local point determining module configured to determine a local set of point from the point cloud based on the predetermined angle; and a local maximum determining module configured to determine a local maximum distance between the local set of points and the centroid; wherein the step angle is determined according to the predetermined angle, the global maximum distance and the local maximum distance.

In some example embodiments, the device further comprising: a coordinate determining module configured to determine a coordinate system in which the point cloud is positioned; an origin determining module configured to determine an origin of the coordinate system; and a translating module configured to translate the point cloud in the coordinate system to allow the centroid to coincide with the origin.

In some example embodiments, the device further comprising: a denoising module configured to denoise the point cloud before determining a centroid of the point cloud.

In some example embodiments, the device further comprising: a splitting module configured to split the point cloud from a background.

In some example embodiments, the device further comprising: a projecting module configured to project the point cloud onto a two-dimensional plane to form a projection; a fitting module configured to fit the projection with a spline curve; a slope calculating module configured to calculate a slope of each points on the spline curve; and a line calculating module configured to calculate an equidistant line based on the slope.

In a third aspect, a computer program product is proposed. The computer program product is tangibly stored in a non-transitory computer storage medium and includes machine-executable instructions, the machine-executable instructions, when executed by a device, causing the device to execute the method of the first aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

Through the following detailed description with reference to the accompanying drawings, the above and other objectives, features and advantages of the example embodiments disclosed herein will become more comprehensible. In the drawings, several example embodiments disclosed herein will be illustrated in an exemplary and in a non-limiting manner, wherein:

Fig. 1 illustrates an example scenario in which the embodiment of the present disclosure may be implemented;

Fig. 2 illustrates a flowchart of a method of generating a path in accordance with an example embodiment of the present disclosure;

Fig. 3 illustrates a flowchart of a method of dynamically adjusting the step angle θ in accordance with an example embodiment of the present disclosure;

Fig. 4 illustrates an example subset of point cloud to describe the step of dynamically adjusting the step angle

Fig. 5 illustrates an example embodiment of the present disclosure, which forms the final line; and

FIG. 6 is a schematic diagram illustrating a device that may be used to implement embodiments of the present disclosure..

Throughout the drawings, the same or similar reference numerals represent the same or similar element.

DETAILED DESCRIPTION

Principles of the present disclosure will now be described with reference to some example embodiments. It is to be understood that these embodiments are described only for the purpose of illustration and to help those skilled in the art to understand and implement the present disclosure, without suggesting any limitation as to the scope of the disclosure. The disclosure described herein can be implemented in various manners other than the ones described below.

In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

References in the present disclosure to “one embodiment, ” “an embodiment, ” “an example embodiment, ” and the like indicate that the embodiment described may include a particular feature, structure, or characteristic, but it is not necessary that every embodiment includes the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to apply such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

It should be understood that although the terms “first” and “second” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and similarly, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, the term “and/or” includes any and all combinations of one or more of the listed terms.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms “a” , “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” , “comprising” , “has” , “having” , “includes” and/or “including” , when used herein, specify the presence of stated features, elements, and/or components etc., but do not preclude the presence or addition of one or more other features, elements, components and/or combinations thereof.

Currently, automatic solutions for sole glue spraying can be divided into three categories. The first category is to model shoes and spray with CAD models, the second category is to generate the sole glue spraying track through teaching, and the third type is to obtain three-dimensional data with the aid of visual sensors, and generate the position information of the sole glue spraying track after processing the three-dimensional point cloud. Among these kinds of methods, the first category of modeling course is complicated, and the actual shoe shape and CAD model processed in simultaneously often have deviation, which affects the accuracy. The second category of method is faced with the difficulty of adapting to different shoe types and cannot be used for mixed production of multiple shoe types. Therefore, there is need to improve the spray of the glue on the shoe sole.

At least to address the problem existed in the conventional approaches, the present disclosure proposes a solution for generating shoe sole spraying path based on local feature analysis for point cloud .

Example embodiments will be described in more detail hereinafter in accordance with Figs. 1-6.

With reference to Figs. 1-2 at first, wherein Fig. 1 illustrates an example scenario in which the embodiment of the present disclosure may be implemented, and Fig. 2 illustrates a flowchart of a method 200 of generating a path in accordance with an example embodiment of the present disclosure. The method 200 may be carried out by a processor for use with a glue spraying device.

At block 202, a point cloud is received. The point cloud comprises a plurality of points indicating an outline of an object. In some example embodiments, the object may be a shoe sole. This is only for illustration without suggesting any limitations as to the scope of the subject matter described here, in other example embodiments, the object may be other objects.

At block 204, a centroid C of the point cloud is determined. The determination of the centroid C may be carried out by any known methods or the methods to be developed in the future. In some example embodiments, a coordinate system in which the point cloud is positioned is determined. After this, the origin of the coordinate system is determined and the point cloud is translated within the coordinate system to allow the centroid to coincide with the origin of the coordinate. In this manner, the calculation in later process may be much easier.

At block 206, a first point P1 is selected from the point cloud as a starting point. At block 208, a second point P2 adjacent to the first point P1 is determined based on a step angle θ. The step angle θ corresponds to the central angle of the object around the centroid C and may be determined by different approaches, which will be described in more details hereinafter. After the points are determined one by one in a round throughout the point cloud, the path of the object can be generated.

In some example embodiments, the step angle θ may be a constant value. For example, if it is required to finally obtain 100 points in a whole, then the step angle θ may be 360° /100 = 3.6°. By providing the step angle θ to be a constant value, the determination of the set of point would be much easier. In the scenario where the baby shoe sole is manufactured, since the baby shoe sole is less elongated than the shoe sole of an adult, such a manner can improve the calculating speed.

In some further example embodiments, in the scenario where the adult shoe sole is manufactured, since the adult shoe sole is relatively elongated than the shoe sole of an adult, in order to improve the calculation accuracy, the step angle θ may be dynamically adjusted when traversing the point cloud. Fig. 3 illustrates a flowchart of a method 300 of dynamically adjusting the step angle θ in accordance with an example embodiment of the present disclosure; and Fig. 4 illustrates an example subset of point cloud to describe the step of dynamically adjusting the step angle θ.

In order to accurately adjusting the step angle θ, a global maximum distance is determined. In some example embodiments, each distance between each point of the point cloud and the centroid C may be calculated. The global maximum distance D _gm may be determined according to each distance.

Referring to Fig. 3, at block 302, a predetermined angle θ _P is determined and at block 304, a local set of point is determined from the point cloud based on the predetermined angle θ _P.

Referring to Fig. 4, the local set of point includes the points Q1, Q2, …, Qn starting from P1 within the central angle having the value of the predetermined angle θ _P.

Then, as can be seen from Fig. 3 at block 306 and Fig. 4, the local maximum distance D _lm between the local set of points and the centroid is determined. At block 308, the step angle θ is determined according to a predetermined angle θ _P, the global maximum distance D _gm and the local maximum distance D _lm.

In the scenario of manufacturing the shoe sole, the global maximum distance D _gm may often exist around the toe or the heel of the shoe sole, which has a relatively curved outline. Since global maximum distance D _gm the In some example embodiments, the step angle θ may be calculated by the following formula:

This is based on the concept that where the local maximum distance D _lm is close to the global maximum distance D _gm, i.e.,

is slightly greater than 1, which means the point is adjacent to the toe or the heel of the shoe sole, and the points here should be denser. That is, the step angle θ should be smaller. On the other hand, where the local maximum distance D _lm is far away from the global maximum distance D _gm, i.e.,

is much greater than 1, that means the point is far away from the toe or the heel of the shoe sole, and the points here should be sparser. That is, the step angle θ should be greater.

According to the method 300, the equal arc length distribution along the path can be realized. This is particularly suitable for the adult shoe soles.

In some example embodiments, the method 200 may further comprise denoising the point cloud before determining a centroid of the point cloud. In this way, the centroid of the point cloud may be calculated more accurately.

In some example embodiments, the method 200 may further comprise splitting the point cloud from a background. In this way, the centroid of the point cloud may be calculated more accurately.

Fig. 5 illustrates an example embodiment of the present disclosure, which forms the final line. In the illustrated embodiments, the method 200 may further comprise projecting the point cloud onto a two-dimensional plane to form a projection; fitting the projection with a spline curve L1; calculating a slope of each point on the spline curve; and calculating an equidistant line based on the slope. The equidistant line forms the final outline L2. Since the final outline L2 is located inwardly from the spline curve L1 by a certain distance, this ensures that if the glue is applied along the final outline L2, the glue would not be spilled out.

In some example embodiments, the method 200 may further comprise mapping the equidistant line back to the initial cloud point and computing the normal vectors for each point of the equidistant line. In this way, the path can be generated. After generating the path, the glue may be sprayed to the shoe sole automatically along the path. In some example embodiments, the spraying procedure may be carried out by an industrial robot.

In another aspect, an electronic device for determining a path is proposed. The device comprising: a receiving module configured to receive a point cloud, the point cloud indicating an outline of an object; a centroid determining module configured to determine a centroid of the point cloud; a selecting module configured to select a first point from the point cloud; and a point determining module configured to determine a second point adjacent to the first point based on a step angle.

In some example embodiments, the step angle is constant.

In some example embodiments, the device may further comprise a mapping module configured to map the equidistant line back to the cloud point and a computing module configured to compute the normal vectors for each point of the equidistant line. In this way, the path can be generated.

Compared to the current approaches, the solution proposed herein is suitable for edge extraction of multiple shoe shapes, and can be applied to mixed production of multiple shoe shapes. Besides, this solution is not sensitive to noise and other disturbances and can reduce the cost of visual sensors in the automatic glue spraying system.

It is to be understood that even though the example embodiments are described in the scenario of manufacturing a shoe sole, the embodiments described herein may also be applied in other scenarios.

FIG. 6 is a schematic diagram illustrating a device 600 that may be used to implement embodiments of the present disclosure. As illustrated, the device 600 includes a central processing unit (CPU) 601, which may execute various appropriate actions and processing based on the computer program instructions stored in a read-only memory (ROM) 602 or the computer program instructions loaded into a random access memory (RAM) 603 from a storage unit 608. The RAM 603 also stores all kinds of programs and data required by operating the storage device 600. CPU 601, ROM 602 and RAM 603 are connected to each other via a bus 604 to which an input/output (I/O) interface 605 is also connected.

A plurality of components in the device 600 are connected to the I/O interface 605, including: an input unit 606, such as keyboard, mouse and the like; an output unit 607, such as various types of displays, loudspeakers and the like; a storage unit 608, such as the magnetic disk, optical disk and the like; and a communication unit 609, such as network card, modem, wireless communication transceiver and the like. The communication unit 609 allows the device 600 to exchange information/data with other devices through computer networks such as Internet and/or various telecommunication networks.

Each procedure and processing described above may be executed by a processing unit 601. For example, in some embodiments, the method may be implemented as computer software programs, which are tangibly included in a machine-readable medium, such as storage unit 608. In some embodiments, the computer program may be partially or completely loaded and/or installed to the device 600 via ROM 602 and/or the communication unit 609. When the computer program is loaded to RAM 603 and executed by CPU 601, one or more steps of the above described method 200 are implemented.

In some embodiments, the

methods

200, 300 described above may be implemented as a computer program product. The computer program product may include a computer-readable storage medium loaded with computer-readable program instructions thereon for executing various aspects of the present disclosure.

The computer-readable storage medium may be a tangible device capable of holding and storing instructions used by the instruction-executing device. The computer-readable storage medium can be, but not limited to, for example, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices or any random appropriate combinations thereof. More specific examples (non-exhaustive list) of the computer-readable storage medium include: portable computer disk, hard disk, random-access memory (RAM) , read-only memory (ROM) , erasable programmable read-only memory (EPROM or flash) , static random access memory (SRAM) , portable compact disk read-only memory (CD-ROM) , digital versatile disk (DVD) , memory stick, floppy disk, mechanical coding device, such as a punched card storing instructions or an emboss within a groove, and any random suitable combinations thereof. The computer-readable storage medium used herein is not interpreted as a transient signal itself, such as radio wave or other freely propagated electromagnetic wave, electromagnetic wave propagated through waveguide or other transmission medium (such as optical pulses passing through fiber-optic cables) , or electric signals transmitted through electric wires.

The computer-readable program instructions described herein may be downloaded from the computer-readable storage medium to various computing/processing devices, or to external computers or external storage devices via Internet, local area network, wide area network and/or wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter or network interface in each computing/processing device receives computer-readable program instructions from the network, and forwards the computer-readable program instructions for storage in the computer-readable storage medium of each computing/processing device.

The computer program instructions for executing the operations of the present disclosure may be assembly instructions, instructions of instruction set architecture (ISA) , machine instructions, machine-related instructions, microcodes, firmware instructions, state setting data, or a source code or target code written by any combinations of one or more programming languages including object-oriented programming languages and conventional procedural programming languages. The computer-readable program instructions may be completely or partially executed on the user computer, or executed as an independent software package, or executed partially on the user computer and partially on the remote computer, or completely executed on the remote computer or the server. In the case where a remote computer is involved, the remote computer may be connected to the user computer by any type of networks, including local area network (LAN) or wide area network (WAN) , or connected to an external computer (such as via Internet provided by the Internet service provider) . In some embodiments, the electronic circuit is customized by using the state information of the computer-readable program instructions. The electronic circuit may be a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) for example. The electronic circuit may execute computer-readable program instructions to implement various aspects of the present disclosure.

The computer-readable program instructions may be provided to the processing unit of a general purpose computer, a dedicated computer or other programmable data processing devices to generate a machine, causing the instructions, when executed by the processing unit of the computer or other programmable data processing devices, to generate a device for implementing the functions/actions specified in one or more blocks of the flow chart and/or block diagram. The computer-readable program instructions may also be stored in the computer-readable storage medium. These instructions enable the computer, the programmable data processing device and/or other devices to operate in a particular way, such that the computer-readable medium storing instructions may comprise a manufactured article that includes instructions for implementing various aspects of the functions/actions specified in one or more blocks of the flow chart and/or block diagram.

The computer-readable program instructions may also be loaded into computers, other programmable data processing devices or other devices, so as to execute a series of operational steps on the computers, other programmable data processing devices or other devices to generate a computer implemented process. Therefore, the instructions executed on the computers, other programmable data processing devices or other devices can realize the functions/actions specified in one or more blocks of the flow chart and/or block diagram.

The accompanying flow chart and block diagram present possible architecture, functions and operations realized by the system, method and computer program product according to a plurality of embodiments of the present disclosure. At this point, each block in the flow chart or block diagram may represent a module, a program segment, or a portion of the instruction. The module, the program segment or the portion of the instruction includes one or more executable instructions for implementing specified logic functions. In some alternative implementations, the function indicated in the block may also occur in an order different from the one represented in the drawings. For example, two consecutive blocks actually may be executed in parallel, and sometimes they may also be executed in a reverse order depending on the involved functions. It should also be noted that each block in the block diagram and/or flow chart, and any combinations of the blocks thereof may be implemented by a dedicated hardware-based system for implementing specified functions or actions, or a combination of the dedicated hardware and the computer instructions.

Various embodiments of the present disclosure have been described above, and the above explanation is illustrative rather than exhaustive and is not limited to the disclosed embodiments. Without departing from the scope and spirit of each explained embodiment, many alterations and modifications are obvious for those ordinary skilled in the art. The selection of terms in the text aims to best explain principle, actual application or technical improvement in the market of each embodiment or make each embodiment disclosed in the text comprehensible for those ordinary skilled in the art.

Claims

A method of determining a path, comprising:

receiving a point cloud, the point cloud indicating an outline of an object;

determining a centroid of the point cloud;

selecting a first point from the point cloud; and

determining a second point adjacent to the first point based on a step angle.
The method of Claim 1, wherein the step angle is constant.
The method of Claim 1, further comprising:

determining each distance between each point of the point cloud and the centroid; and

determining a global maximum distance according to each distance.
The method of Claim 3, further comprising:

determining a predetermined angle;

determining a local set of point from the point cloud based on the predetermined angle; and

determining a local maximum distance between the local set of points and the centroid;

wherein the step angle is determined according to the predetermined angle, the global maximum distance and the local maximum distance.
The method of any of Claims 1-4, further comprising:

determining a coordinate system in which the point cloud is positioned;

determining an origin of the coordinate system; and

translating the point cloud in the coordinate system to allow the centroid to coincide with the origin.
The method of any of Claims 1-4, further comprising:

denoising the point cloud before determining a centroid of the point cloud.
The method of any of Claims 1-4, further comprising:

splitting the point cloud from a background.
The method of any of Claims 1-4, further comprising:

projecting the point cloud onto a two-dimensional plane to form a projection;

fitting the projection with a spline curve;

calculating a slope of each points on the spline curve; and

calculating an equidistant line based on the slope.
An electronic device for determining a path comprising:

a receiving module configured to receive a point cloud, the point cloud indicating an outline of an object;

a centroid determining module configured to determine a centroid of the point cloud;

a selecting module configured to select a first point from the point cloud; and

a point determining module configured to determine a second point adjacent to the first point based on a step angle.
The electronic device of Claim 9, wherein the step angle is constant.
The electronic device of Claim 9, further comprising:

a distance determining module configured to determine each distance between each point of the point cloud and the centroid; and

a global maximum determining module configured to determine a global maximum distance according to each distance.
The electronic device of Claim 11, further comprising:

an angle determining module configured to determine a predetermined angle;

a local point determining module configured to determine a local set of point from the point cloud based on the predetermined angle; and

a local maximum determining module configured to determine a local maximum distance between the local set of points and the centroid;

wherein the step angle is determined according to the predetermined angle, the global maximum distance and the local maximum distance.
The electronic device of any of Claims 9-12, further comprising:

a coordinate determining module configured to determine a coordinate system in which the point cloud is positioned;

an origin determining module configured to determine an origin of the coordinate system; and

a translating module configured to translate the point cloud in the coordinate system to allow the centroid to coincide with the origin.
The electronic device of any of Claims 9-12, further comprising:

a denoising module configured to denoise the point cloud before determining a centroid of the point cloud.
The electronic device of any of Claims 9-12, further comprising:

a splitting module configured to split the point cloud from a background.
The electronic device of any of Claims 9-12, further comprising:

a projecting module configured to project the point cloud onto a two-dimensional plane to form a projection;

a fitting module configured to fit the projection with a spline curve;

a slope calculating module configured to calculate a slope of each points on the spline curve; and

a line calculating module configured to calculate an equidistant line based on the slope.
A computer program product, which is tangibly stored in a non-transitory computer storage medium and includes machine-executable instructions, the machine-executable instructions, when executed by a device, causing the device to execute the method of any one of claims 1-8.