SE2250597A1

SE2250597A1 - Multi-sensor evaluation of a printing process

Info

Publication number: SE2250597A1
Application number: SE2250597A
Authority: SE
Inventors: Rahul Suresh
Original assignee: Cellink Bioprinting Ab
Priority date: 2022-05-19
Filing date: 2022-05-19
Publication date: 2023-11-20
Also published as: WO2023224528A1

Abstract

The present disclosure relates to a method, control system, a 3D bioprinter comprising the control system, a computer program carrier and a computer program product. The method comprises obtaining sensor data associated with a plurality of sensor devices configured to obtain sensor detections of the printing process, the sensor data comprising sensor data associated with each sensor device of the plurality of sensor devices. The method further comprises processing the obtained sensor data, and determining a weighted component for the processed data by means of an association weight function in order to create one or more weighted processed data set(s). The method further comprises determining a sum vector based on the one or more weighted processed data set(s) by summing the one or more weighted processed data set(s) and determining a state of the printing process by evaluating the determined sum vector against a print-process threshold value. Even further, the method comprises generating a state-signal indicative of the state of the printing process based on the evaluation of the sum vector against the print-process threshold value.

Description

TITLE I\/lulti-sensor evaluation of a printing process TECHNICAL FIELD The aspects and embodiments of the present disclosure relate to automated assessment of a printing process by a 3D bioprinter. More specifically, the aspects and embodiments of the present disclosure relate to systems and methods for automatically assessing the printing process of the 3D bioprinter by means of sensor devices and sensor data analysis. BACKGROUND During the past few years, the research and development activities related to 3-dimentional (3D)-bioprinting have exploded in number and many different approaches are currently being explored. Amongst many technical and design parameter considerations and implementations of a 3D bioprinting process is the ability to ensure a smooth and seamless flow ofthe printing process. To this end, an important step is to ascertain during the entire printing process that all the parameters and quality metrics are met by the process and by the end product i.e. the printed object. This includes the pre-print phase wherein e.g. adequate priming of a dispensing means such as a dispensing nozzle and/or a dispensing needle of the 3D-bioprinter configured to deliver the printing fluid to the printing bed is to be ensured. Further, during a run-time of the printing process, performing the act of printing of the desired structures is to be monitored and verified. Additionally, in the post-print phase, the printed objects are to be monitored for the integrity and be verified against expected print quality metrics.

Conventionally, the process of verification ofa printing process has been associated with a wide range of problems including unnecessary user involvement in the monitoring and verification procedure, inadequate calibration of the 3D-bioprinter in the pre-print phase, inadequate priming of the bioprinter nozzle, inaccurate assessment of print metrics during pre-print, run- time and post-print phases, increased amounts of material waste per print job or with continuation of the printing process, risks of erroneously allowing a faulty print job to continue beyond an irreparable state, etc.

Accordingly, for elevated comfort, safety and accuracy of a printing process by a 3D-bioprinter, there is a need for solutions in the art that are capable of monitoring and verification of the entire printing process in a robust and reliable manner.

SUMMARY lt is therefore an object ofthe present disclosure to provide a control system, a 3D bioprinter comprising such a control system, a method, a computer program carrier, and a computer program product which alleviate all or at least some of the drawbacks of presently known solutions.

More specifically, it is an object of the present disclosure to alleviate problems related to automated monitoring and verification of a printing process by a 3D bioprinter.

These objects are achieved by means of a control system, a 3D bioprinter comprising such a control system, a method, a computer program carrier, and a computer program product, as defined in the appended independent claims. The term exemplary is in the present context to be understood as serving as an instance, example or illustration.

To monitor and analyze the printing process in a pre-print phase, in real time during the run- time ofthe print job and also in the post-print phase, several parameters such as dispensing nozzle and/or needle priming, print quality, print texture, post-print reactiveness, structural analysis, quality of final print, etc. are to be considered. Accordingly based on the result of such monitoring and analysis decisions may be formed to continue or stop an ongoing printing process, or to change process parameters ofthe 3D bioprinter, adapt the design ofthe desired print object, and the like. The analyzing may also be used for notifying or warning a user of the 3D bioprinter for unforeseen events or if printing reaches a predefined warning situation e.g., a close-to-borders situation, or structure-out-of-limits situation on the print bed.

According to a first aspect of the present disclosure, there is provided a method for assessing a printing process performed by a 3D bioprinter. The method comprises obtaining sensor data associated with a plurality of sensor devices configured to obtain sensor detections of the printing process, the sensor data comprising sensor data associated with each sensor device of the plurality of sensor devices. The method further comprises processing the obtained sensor 3 data. Wherein the step of processing the obtained sensor data comprises any one of analyzing each sensor data associated with each sensor device separately in order to create at least one separately analyzed sensor data point, and fusing the obtained sensor data of at least two sensor devices of the plurality of sensor devices in order to create one or more fused sensor data point(s) and analyzing each ofthe created one or more fused sensor data point(s) in order to create one or more analyzed-fused data point(s), and analyzing each sensor data associated with each sensor device separately in order to create at least one separately analyzed sensor data point and fusing the at least one separately analyzed sensor data point with at least one other separately analyzed sensor data point in order to create one or more fused-analyzed sensor data combination(s), and fusing the at least one separately analyzed sensor data point with at least one of the one or more analyzed-fused data point(s). Additionally, the method comprises determining a weighted component for each ofthe at least one separately analyzed sensor data point, and/or for the one or more analyzed-fused data point(s), and/or for the one or more fused-analyzed sensor data combination(s), and/or for the fused at least one separately analyzed sensor data point with the at least one of the one or more analyzed-fused data point(s) by means of an association weight function in order to create one or more weighted processed data set(s). Even further, the method comprises determining a sum vector based on the one or more weighted processed data set(s) by summing the one or more weighted processed data set(s). The method further comprises determining a state of the printing process by evaluating the determined sum vector against a print-process threshold value and generating a state-signal indicative ofthe state ofthe printing process based on the evaluation of the sum vector against the print-process threshold value. ln some embodiments, the step of determining the state of the printing process may comprise determining ifthe determined sum vector fulfills at least one requirement ofthe printing process, wherein the print-process threshold value may comprise a predetermined threshold value for each of the at least one requirement of the printing process. ln several embodiments, the method may further comprise controlling the printing process based on the generated state-signal. ln various embodiments the printing process may comprise at least one of a pre-print phase, run-time print phase, and a post-print phase. 4 Accordingly, in several embodiments the method may further comprise controlling the printing process by forming a continuation or interruption decision for each one of the phases of the printing process based on the determined state of the printing process.

According to various embodiments and aspects, the at least one requirement ofthe printing process may comprise any one of an allowable calibration of a nozzle ofthe 3D bioprinter, an allowable priming ofthe nozzle of the 3D bioprinter, an allowable state of a dispensing needle connected to the nozzle ofthe 3D bioprinter, an indicator of clogging of the nozzle of the 3D printer, an allowable point tip calibration of the nozzle of the 3D bioprinter, an allowable flowrate and/or a volume, and/or a temperature, and/or a viscosity of a printing fluid being introduced into the nozzle of the 3D bioprinter, and an allowable size and/or a volume of a hanging drop|et of the printing fluid extruded out of the tip of the nozzle ofthe 3D bioprinter and/or at least one quality-of-print metric. ln several embodiments and aspects the plurality of sensor devices may comprise any one of a camera, a thermal camera, a thermal sensor, a lidar, a radar, a laser, a Wheatstone bridge circuit arranged under a print bed ofthe 3D bioprinter, a pressure sensor, an ultrasound sensor, and a microwave radio frequency, RF, sensor. ln several embodiments and aspects, the method may further comprise determining the weighted component, by means of the association weight function, by multiplying a predetermined weight value with each of the at least one separately analyzed sensor data point, and/or with the one or more analyzed-fused data point(s), and/or with the one or more fused-analyzed sensor data combination(s), and/or with the fused at least one separately analyzed sensor data point with the at least one ofthe one or more analyzed-fused data point(s).

According to yet second aspect of the present disclosure there is provided a system for assessing a printing process by a 3D bioprinter. The system comprises processing circuitry configured to obtain sensor data associated with a plurality of sensor devices configured to obtain sensor detections of the printing process, the sensor data comprising sensor data associated with each sensor device of the plurality of sensor devices. Further, the processing circuitry is configured to process the obtained sensor data and analyze each sensor data associated with each sensor device separately in order to create at least one separately analyzed sensor data point, and fuse the obtained sensor data of at least two sensor devices of the plurality of sensor devices in order to create one or more fused sensor data point(s) and analyze each ofthe created one or more fused sensor data point(s) in order to create one or more analyzed-fused data point(s), and analyze each sensor data associated with each sensor device separately in order to create at least one separately analyzed sensor data point and fuse the at least one separately analyzed sensor data point with at least one other separately analyzed sensor data point in order to create one or more fused-analyzed sensor data combination(s), and fuse the at least one separately analyzed sensor data point with at least one of the one or more analyzed-fused data point(s). The processing circuitry is further configured to determine a weighted component for each of the at least one separately analyzed sensor data point, and/or for the one or more analyzed-fused data point(s), and/or for the one or more fused-analyzed sensor data combination(s), and/or for the fused at least one separately analyzed sensor data point with the at least one of the one or more analyzed- fused data point(s) by means of an association weight function in order to create one or more weighted processed data set(s). ln addition the processing circuitry is configured to determine a sum vector based on the one or more weighted processed data set(s) by summing the one or more weighted processed data set(s). further, the processing circuitry is configured to determine a state ofthe printing process by evaluating the determined sum vector against a print-process threshold value and generate a state-signal indicative of the state ofthe printing process based on the evaluation ofthe sum vector against the print-process threshold value.

According to a further third aspect, there is provided a computer program carrier carrying one or more computer programs configured to be executed by one or more processors of a processing circuitry, the one or more programs comprising instructions for performing the method according to any one ofthe embodiments of the method herein, and wherein the computer program carrier is one of an electronic signal, optical signal, radio signal or a computer-readable storage medium.

According to yet another fourth aspect, there is provided a computer program product comprising instructions which, when the program is executed by one or more processors of a processing circuitry, causes the processing circuitry to carry out the method according to any one ofthe embodiments of the method herein. 6 According to a fifth aspect, there is provided a 3D bioprinter comprising one or more print heads, each print head comprising a dispensing nozzle having an in|et for introduction of a printing fluid into the dispensing nozzle and an outlet for extruding the printing fluid from the outlet ofthe dispensing nozzle; and a system according to any one of embodiments of the system herein for assessing the printing process performed by the 3D bioprinter. Further embodiments of the different aspects are defined in the dependent claims. lt is to be noted that all the embodiments, elements, features and advantages associated with the first aspect also analogously apply to the second, third, fourth and fifth aspects of the present disclosure.

These and other features and advantages of the present disclosure will in the following be further clarified in the following detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS Further objects, features and advantages of embodiments of the disclosure will appear from the following detailed description, reference being made to the accompanying drawings. The drawings are not to scale.

Fig. 1 is a schematic block diagrams illustrating various embodiments of a system for assessing a printing process according to the present disclosure; Fig. 2 is a schematic flowchart illustrating a method in accordance with several embodiments of the present disclosure; and Fig. 3 is a schematic front view illustration of a 3D bioprinter comprising the control system in accordance with an embodiment of the present disclosure. DETAILED DESCRIPTION Those skilled in the art will appreciate that the steps, services and functions explained herein may be implemented using individual hardware circuitry, using software functioning in conjunction with a programmed microprocessor or general purpose computer, using one or more Application Specific Integrated Circuits (ASICs) and/or using one or more Digital Signal Processors (DSPs). lt will also be appreciated that when the present disclosure is described in 7 terms of a method, it may also be embodied in one or more processors and one or more memories coupled to the one or more processors, wherein the one or more memories store one or more programs that perform the steps, services and functions disclosed herein when executed by the one or more processors. ln the following description of exemplary embodiments, the same reference numerals denote the same or similar components.

Fig. 1 illustrates a schematic block diagram of a plurality of sensor devices 6a-6n, collectively referred to as a sensing unit 6. The sensor devices 6a-6n are adapted to acquire information from the printing process performed by a 3D bioprinter 100. ln various aspects and embodiments, the sensing unit 6 comprises different types of sensor devices 6a-6n which may be any one of or any multi-sensor combination of imaging sensor devices such as a camera like a fixed-focused camera or an auto-focus camera, or other types of sensor devices such as a thermal camera, a thermal sensor, a lidar, a radar, a laser, a Wheatstone bridge circuit arranged under a print bed 101 of the 3D bioprinter 100, a pressure sensor, an ultrasound sensor, and a microwave radio frequency, RF, sensor, etc. lt should be appreciated that the sensor devices 6a-6n may be physically collocated in the sensing unit 6 or in several embodiments be placed in different locations with respect to each other, yet being referred to as the sensing unit 6 collectively. The sensing unit 6 comprising the one or more sensor devices 6a-6n is configured to, based on the defined functionality of each sensor device, acquire information from various components, units, or objects involved in the printing process. For instance, the sensing unit 6 may be configured to acquire information from one or more units or modules ofthe 3D bioprinter such as position or elevation of one or more printheads 1 ofthe 3D bioprinter 100.

Further, the sensing unit 6 is configured to acquire information from object under print 110 which may be referred to as the print object 110 such as various parameters associated with the structural, mechanical, chemical, thermal, optical properties ofthe print object 110 during various phases of the printing process such as during the run-time phase or in post-print phase. Further, the sensing unit 6 may be configured to acquire information from a dispensing nozzle 2 of each printhead 1 such as various parameters of a printing fluid being present in and to be extruded from the dispensing nozzle 2. The nozzles and/or needles ofthe printhead 8 1 may be used interchangeably throughout this description. ln some embodiments a nozzle 2 may be ofthe type of a dispensing needle 2 or the nozzle may comprise a needle as a part of the nozzle.

The printing fluid may be comprised in the printhead 1 e.g. stored in a container (not shown) of the printhead 1 and be supplied to the respective nozzles 2 ofthe one or more printheads 1 to be extruded out of the nozzles 2 during various phases of the printing process such as priming and print job. The printing fluid may comprise any one of a bioink, a printing ink, a hydrogel, a chemical reagent, a cell medium, crosslinking agents, and a gel. Examples of bioink may comprise any one of collagen, ColMA, GelMA, GelXA, alginate, pluronic, nanofibrillated cellulose, HAMA, Cellink, etc. ln this description, when using printing fluid it should be appreciated that it encompasses any one, or any combination of the above examples of the printing fluid as well as embodiments where external objects may be present in the printing fluid to be printed. External objects present or mixed into the printing fluid may for instance be any one of at least one living cell, at least one bead such as spherical plastic beads, fluorescent beads used for tracing, etc., at least one organoid i.e. aggregates of living cells, and at least one spheroid i.e. spherically-shaped aggregates of living cells, etc. the sensing unit 6 may also be configured to acquire information regarding the external objects present in the printing fluid, while the printing fluid being extruded from the nozzle 2, under the print job phase i.e. when the print object 110 is being constructed and/or after the print object 110 is pa rtly or fully constructed.

According to several embodiments and aspects, the sensing unit 6 is configured to obtain raw sensor measurements from the different components and objects as explained above and transmit the obtained raw sensor measurements to processing circuitry 11 adapted to process the raw sensor measurements of the plurality of sensor devices 6a-6n and obtain associated sensor data. The raw measurements may in some embodiments be pre-processed for instance go through steps of filtering, amplification, noise reduction etc. ln some embodiments raw sensor data may be used directly. ln some embodiments and examples a combination of pre- processed sensor data and raw sensor data may be used by the processing circuitry 11. Accordingly, it should be understood that parts of the described solution, particularly a system 10 may be implemented either in the 3D bioprinter 100 itself, in a system located external the 3D bioprinter 100, or in a combination of internal and external the 3D bioprinter; for instance 9 in an external server or remote control center (not shown) in communication with the 3D bioprinter 100. The solution may in several embodiments be fully or partly implemented on external networks 20 such as on a cloud platform 20.

The system 10 which may also be referred to as a control system 10 or a control device 10 comprises processing circuitry 11 configured to perform assessing ofthe printing process by the 3D bioprinter 100 in accordance with several aspects and embodiments of the present disclosure. The processing circuitry 11 is configured to obtain sensor data associated with the plurality of sensor devices 6a-6n configured to obtain sensor detections of the printing process, the sensor data comprising sensor data associated with each sensor device of the plurality of sensor devices. Several types of sensor devices including cameras, thermal cameras, thermal sensors, Lidars, Radars, lasers, Wheatstone bridge circuit under the print bed, pressure sensors, ultrasound, microwaves, etc. can be used for acquiring measurements.

As shown in Fig. 1, the sensor measurements from the sensing unit 6 is provided to the system 10 comprising processing circuitry 11. The processing circuitry 11 may be configured to at least partly pre-process the raw measurements and obtain sensor data associated with the plurality of the sensor devices 6a-6n. The obtained sensor data may be provided to a data processing unit 11a of the system 10. ln several embodiments, the data processing unit 11a may be implemented as a part of or as a functionality ofthe processing circuitry 11 or be a separate data processing unit in communication with and subordinate to the processing circuitry 11. The data processing unit 11a may also be referred to as data fusion unit 11a. The processing circuitry 11 is configured to process the obtained sensor data from the sensing unit 6. The data processing according to several aspects and embodiments comprises analyzing each sensor data associated with each sensor device 6a-6n separately in order to create at least one separately analyzed sensor data point. The data processing also comprises fusing the obtained sensor data of at least two sensor devices 6a-6n of the plurality of sensor devices 6a-6n in order to create one or more fused sensor data point(s) and analyzing each ofthe created one or more fused sensor data point(s), thus creating analyzed-fused sensor data point(s). ln the present context, by analyzed-fused data points(s) it is meant to convey that the sensor data of the at least two sensor devices is combined (fused) prior to the analysis step, thus it is the fused sensor data which is analyzed by the processing circuitry.

Further, the processing circuitry 11 is configured to fuse the at least one separately analyzed sensor data point with at least one other separately analyzed sensor data point in order to create one or more fused-analyzed sensor data combination(s). ln the present context, by fused-analyzed data combination(s) it is meant to convey that one or more com bination(s) (data fusions) of separately analyzed sensor data are created by the processing circuitry.

Even further, the processing circuitry 11 is configured to fuse the at least one separately analyzed sensor data point with at least one of the one or more analyzed-fused data point(s) in order to create one or more fused at least one separately analyzed sensor data point with the at least one ofthe one or more analyzed-fused data point(s). ln other words, the sensor data associated with each of the sensor devices may be used on its own and be analyzed separately or it can be combined with sensor data from other sensor devices to create fused sensor data points. For instance sensor data associated with one or more cameras may be fused with the sensor data from one or more radar sensor devices to create a data representation such as point-cloud representation or pooled depth map of the obtained sensor data for instance created by a data representation unit 11b or data representation functionality 11b of the processing circuitry 11 as shown in Fig. 1. Similar data representations may be created for any sensor data type obtained from various sensor devices. The created data representations may later be further analyzed by the processing circuitry for assessment ofthe printing process as will be explained in more detail below.

Not only the obtained sensor data of at least two sensor devices may be fused but also the separately analyzed data points from single sensor devices may be fused together. This way different data types such as vision-based data types may be combined with pressure or temperature data types which cannot be directly fused together however, but can be analyzed separately and processed in fused analyzed sensor data combination(s).

Thus, the data fusion may be combination of two different set of data types being fused and analysed (like a signal and an image analyzed separately) and/or may be combination of data obtained from two different types of sensors to create a new set of fused data points and analyzing the fused data points (e.g. 3D mapping from a Lidar sensor and image data from a camera). 11 ln several aspects and embodiments, not only the obtained sensor data acquired from online sensor measurements by the sensing unit 6 is used in the data processing but also offline sensor data from previous sensors measurements which for instance may be stored on a cloud network may be used to create the analyzed and fused data points. lt should be noted that in the rest of this description when referring to a data point or data set as fused data point or fused data set it is to be construed as any of the different ways of combining and analyzing data obtained from the printing process regardless ofthe data type. ln some exemplary embodiments wherein imaging sensor devices such as cameras are used as primary sensor devices, raw image measurements may be obtained from various objects and components e.g. the nozzle 2 or the print object 110 and the raw image data obtained by the imaging sensor device may be processed by means of an image processing algorithm. Vision- based data fusion techniques such as SLAM (Simultaneous localization and mapping) can then be applied to the obtained associated sensor data from the imaging sensor devices.

Accordingly, as realized by the present inventor, the proposed solution makes it possible that data retrieved from multiple sensor devices, regardless of the type of the sensor device, work in synchrony and interpretation of data will be aligned and equal in the analysis part of the system.

When the associated sensor data is processed by the system 10, and the data representation is created, the processing circuitry 11 is configured to determine a weighted component for each ofthe at least one separately analyzed and/or fused data by means of an association weight function in order to create one or more weighted processed data set(s). The fused data as mentioned earlier comprises the one or more analyzed-fused data point(s) and/or one or more fused-analyzed sensor data combination(s). The processing circuitry 11 is further configured to determine a sum vector, i.e. addition vector, based on the one or more weighted processed data set(s) by summing the one or more weighted processed data set(s). in some embodiments the weighted sum might be predefined. Accordingly, the processing circuitry may be configured to determine the weighted component, by means of the association weight function, by multiplying a predetermined weight value with each of the at least one analyzed and/or one or more analyzed-fused data point(s) and/or one or more fused-analyzed sensor data combination(s). 12 By creating such a vectorized representation of the sensor data, the processing circuitry is adapted to determine a state of the printing process by evaluating the determined sum vector against a print-process threshold value and generate a state-signal indicative ofthe state of the printing process based on the evaluation of the sum vector against the print-process threshold value. The processing circuitry may further be configured to control the printing process based on the generated state-signal. ln several aspect and embodiments, the processing circuitry may be further configured to determine the state ofthe printing process by determining ifthe determined sum vector fulfills at least one requirement of the printing process, wherein the print-process threshold value comprises a predetermined threshold value for each ofthe at least one requirement of the printing process. ln other words, the processing circuitry is configured to compare the vectorized representation ofthe sensor data from the printing process with an assessment criteria which comprises requirements of the printing process to be fulfilled. Each ofthese requirements are accordingly associated with a respective predetermined threshold value. Thus, the processing circuitry 11, is adapted to compare the sum vector with each ofthe predetermined threshold values in order to determine if the printing processes fulfills the requirements.

As also mentioned earlier, the printing process may comprise at least one of a pre-print phase, run-time print phase (i.e. the printjob), and a post-print phase. ln several embodiments and aspects, the at least one requirement of the printing process may comprise any one of an allowable calibration of a nozzle ofthe 3D bioprinter, an allowable priming of the nozzle ofthe 3D bioprinter, an allowable state of a dispensing needle connected to the nozzle ofthe 3D bioprinter, an indicator of clogging of the nozzle of the 3D printer, an allowable point tip calibration of the nozzle ofthe 3D bioprinter, an allowable flowrate and/or a volume, and/or a temperature, and/or a viscosity of a printing fluid being introduced into the nozzle ofthe 3D bioprinter, and an allowable size and/or a volume of a hanging droplet ofthe printing fluid extruded out ofthe tip of the nozzle of the 3D bioprinter. Even further, in some embodiments the at least one requirement ofthe printing process may further comprise at least one quality- of-print metric for the print object 110 during the run-time print phase and/or in the post- print phase. Various parameters such as print quality, print texture, post-print reactiveness, 13 structural analysis, print uniformity, continuity, and regularity, air bubbles in bioink, etc. of the print object 110 may be comprised in the quality-of-print metrics. ln several embodiments and aspects the processing circuitry may be further configured to control the printing process by forming a continuation or interruption decision for each one of the phases of the printing process based on the determined state of the printing process. For instance, if during the print job the system 10 determines that an error has occurred in the printing process and the print object 110 does not meet the requirements of the quality-of- print metrics or if there have been irregularities in the position, volume, viscosity, temperature or any other parameters of the printing fluid, the system 10 may terminate the print job and further alert the user of the 3D bioprinter of the unforeseen event e.g. by displaying error signals on a display 102 of the 3D bioprinter 100 or by producing sound alarm signals. Similarly, if at a post-print phase, the system 10 determines that the finished print object 110 does not meet the quality-of-print metrics, the system 10 may terminate any subsequent print jobs based on the current print parameters and also alert the user to reevaluate and/or update the parameters ofthe printing process.

Fig. 2 shows a flow chart of a method 300 according to various embodiments and aspects of the present disclosure for assessing a printing process by a 3D bioprinter 100. All the elements, features, and advantages explained in relation to the other aspects also apply analogously to this aspect of the present disclosure.

The method comprises obtaining 301 sensor data associated with a plurality of sensor devices 6a-6n configured to obtain sensor detections of the printing process, the sensor data comprising sensor data associated with each sensor device of the plurality of sensor devices.

The method further comprises processing 303 the obtained sensor data, wherein the processing comprises any one of analyzing 305 each sensor data associated with each sensor device separately in order to create at least one separately analyzed sensor data point, and fusing 307 the obtained sensor data of at least two sensor devices of the plurality of sensor devices in order to create one or more fused sensor data point(s), and analyzing 309 each of the created one or more fused sensor data point(s) in order to create one or more analyzed- fused data point(s), and analyzing each sensor data associated with each sensor device separately in order to create at least one separately analyzed sensor data point and fusing 311 14 the at least one separately analyzed sensor data point from step 305 with at least one other separately analyzed sensor data point from step 305 in order to create one or more fused- analyzed sensor data combination(s)and fusing 312 the at least one separately analyzed sensor data point with at least one of the one or more analyzed-fused data point(s) in order to create one or more fused at least one separately analyzed sensor data point with the at least one of the one or more analyzed-fused data point(s). By the fused at least one separately analyzed sensor data point with the at least one ofthe one or more analyzed-fused data point(s) in the present context it is meant that the outcome of one or more ofthe at least one separately analyzed sensor data points may be fused with the outcome ofthe at least one of the one or more created analyzed-fused data point(s) in order to generate a new dimension of fused data sets.

When the obtained sensor data is processed by the processing circuitry 11, the analyzed data points and/or created data representations are subjected to further processing by the processing circuitry 11 using simple logic and/or neural networks. Thus the method 300 further comprises determining 313 a weighted component for each of the at least one separately analyzed sensor data point and/or one or more analyzed-fused data point(s) and/or one or more fused-analyzed sensor data combination(s) and/or the fused at least one separately analyzed sensor data point with the at least one ofthe one or more analyzed-fused data point(s) by means of an association weight function in order to create one or more weighted processed data set(s). After creating the weighted components, the method further comprises determining 315 a sum vector based on the one or more weighted processed data set(s) by summing the one or more weighted processed data set(s). Further, the method comprises determining 317 a state of the printing process by evaluating 318 the determined sum vector against a print-process threshold value. When the vectorized sensor data is produced and evaluated against the print-process threshold value i.e. compared with the print-process threshold value, the method further comprises generating 319 a state-signal indicative ofthe state ofthe printing process based on the evaluation of the addition vector against the print-process threshold value. ln other words, the result of the comparison of the sum vector with the print-process threshold value generates an output which is used to assess the printing process. ln some embodiments, the method 300 may further comprise determining 317 the state of the printing process by determining 321 if the determined sum vector fulfills at least one requirement 200 of the printing process, wherein the print-process threshold value comprises a predetermined threshold value for each of the at least one requirement ofthe printing process. Thus, the sum vector is compared with all of the print-process threshold values of the one or more requirement(s) of the printing process to determining the state of the printing process i.e. assessing the printing process. ln several embodiments the method may further comprise controlling 323 the printing process based on the generated state-signal. ln various embodiments, the printing process may comprise at least one of a pre-print phase, run-time print phase, and a post-print phase. ln some embodiments the method may comprise controlling 323 the printing process by forming 325 a continuation or interruption decision which may be a binary decision for each one of the phases ofthe printing process based on the determined state ofthe printing pFOCeSS. ln several embodiments, the at least one requirement of the printing process may comprise any one of an allowable calibration of a nozzle of the 3D bioprinter, an allowable priming of the nozzle of the 3D bioprinter, an allowable state of a dispensing needle connected to the nozzle of the 3D bioprinter, an indicator of clogging ofthe nozzle of the 3D printer, an allowable point tip calibration ofthe nozzle of the 3D bioprinter, an allowable flowrate and/or a volume, and/or a temperature, and/or a viscosity of a printing fluid being introduced into the nozzle of the 3D bioprinter, and an allowable size and/or a volume of a hanging droplet of the printing fluid extruded out of the tip ofthe nozzle ofthe 3D bioprinter and/or at least one quality-of-print metric. ln several embodiments, the plurality of sensor devices may comprise any one of a camera, a thermal camera, a thermal sensor, a lidar, a radar, a laser, a Wheatstone bridge circuit arranged under a print bed of the 3D bioprinter, a pressure sensor, an ultrasound sensor, and a microwave radio frequency, RF, sensor. 16 ln various embodiments, the method may further comprise determining 313 the weighted component, by means of the association weight function, by multiplying 327 a predetermined weight value with each of the at least one separately analyzed sensor data point and/or one or more analyzed-fused data point(s) and/or one or more fused-analyzed sensor data combination(s) and/or the fused at least one separately analyzed sensor data point with the at least one ofthe one or more analyzed-fused data point(s).

Executable instructions for performing the above functions and features of the embodiments of the methods may, optionally, be included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors of the processing circuitry.

I\/|ore specifically, there is provided a computer program carrier carrying one or more computer programs configured to be executed by one or more processors of a processing circuitry, the one or more programs comprising instructions for performing any one of the embodiments of the method 300 according to this disclosure. The computer program carrier may be one of an electronic signal, optical signal, radio signal or a computer-readable storage medium.

Even further, there is provided a computer program product comprising instructions which, when the program is executed by one or more processors of a processing circuitry, causes the processing circuitry to carry out any one of the embodiments of the method 300 according to this disclosure.

Fig. 3 is a schematic front view illustration of a 3D bioprinter 100 according to several aspects and embodiments ofthe present disclosure. The 3D bioprinter comprises one or more printheads 1, wherein each printhead 1 comprising a dispensing nozzle 2 having an inlet for introduction of a printing fluid into the dispensing nozzle and an outlet for extruding the printing fluid from the outlet of the dispensing nozzle 2. The 3D bioprinter 100 may comprise a sensing unit 6 comprising one or more sensor devices 6a-6n configured to obtain various types of sensor detection measurements ofthe dispensing nozzles, printing fluid, the print bed 101 ofthe 3D bioprinter, the print object 110, etc. during various phases of the printing process i.e. pre-print phase, run-time phase, post-print phase. ln some embodiments, the 3D bioprinter 100 may be provided with the sensing unit 6 as a separate entity being in communication with the 3D bioprinter 100. The 3D bioprinter also comprises a system 10 17 configured to perform any one of the embodiments of the method 300 according to this disclosure for assessment of the printing process performed by the 3D bioprinter.

The control system 10 may further comprise one or more processors 11, a memory 8, a sensor interface 13 and a communication interface 14. The processor(s) 11 may also be referred to as a control circuit 11 or control circuitry 11 or processing circuitry 11. The control circuitry 11 is configured to execute instructions stored in the memory 8 to perform a method 300 for assessing the printing process according to any one of the embodiments disclosed herein. The memory 8 of the control system 10 can include one or more (non-transitory) computer- readable storage mediums, for storing computer-executable instructions, which, when executed by one or more computer processors 11, for example, can cause the computer processors 11 to perform the techniques described herein. The memory 8 optionally includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.

Further, the 3D bioprinter 100 may be connected to external network(s) 20 via for instance a wireless link or communication interface 14 via various technologies such as cellular long range or short range such as Wireless Local Area (LAN), WiFi, etc. communication technologies.

The present disclosure has been presented above with reference to specific embodiments. However, other embodiments than the above described are possible and within the scope of the disclosure. Different method steps than those described above, performing the method by hardware or software, may be provided within the scope of the disclosure.

For instance, according to an exemplary embodiment a cloud computing system 20 can be configured to perform any one of or any combination ofthe embodiments of the method 300 presented herein. The cloud computing system may comprise distributed cloud computing resources that jointly perform the methods presented herein under control of one or more computer program products. 18 The processor(s) 11 (associated with the control system 10) may be or include any number of hardware components for conducting data or signal processing or for executing computer code stored in memory 8. The system 10 may have an associated memory 8, and the memory 8 may be one or more devices for storing data and/or computer code for completing or facilitating the various methods described in the present description. The memory may include volatile memory or non-volatile memory. The memory 8 may include database components, object code components, script components, or any other type of information structure for supporting the various activities of the present description. According to an exemplary embodiment, any distributed or local memory device may be utilized with the systems and methods of this description. According to an exemplary embodiment the memory 8 is communicably connected to the processor 11 (e.g., via a circuit or any other wired, wireless, or network connection) and includes computer code for executing one or more of processes described herein. lt should be appreciated that the 3D bioprinter 100 may further comprise a sensor interface 13 which may also provide the possibility to acquire sensor data directly or via dedicated sensor control circuitry. The sensor devices 6a-6n such as imaging sensor devices e.g. camera sensors may communicate with the control system 10 using a local network setup, Ethernet, optical fibres, and so on.

As used herein, the term "if" may be construed to mean "when or "upon" or "in response to determining or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined' or "when it is determined" may be construed to mean "upon determining or "in response to determining" or "upon detecting and identifying occurrence of an event" or "in response to detecting occurrence of an event" depending on the context. The term "obtaining" is herein to be interpreted broadly and encompasses receiving, retrieving, collecting, acquiring, and so forth directly and/or indirectly between two entities configured to be in communication with each other or with other external entities.

Accordingly, it should be understood that parts ofthe described methods and systems may be implemented either in the 3D bioprinter 100, in a system located external the 3D bioprinter 100, or in a combination of internal and external the 3D printer; for instance in a server in communication with the 3D printer, e.g. a so called cloud solution 20. For instance, sensor 19 data may be sent to an external system (not shown) and that system performs the steps ofthe methods herein. The different features and steps ofthe embodiments may be combined in other combinations than those described. lt should be noted that the word "comprising" does not exclude the presence of other elements or steps than those listed and the words "a" or "an" preceding an element do not exclude the presence of a plurality of such elements. lt should further be noted that any reference signs do not limit the scope ofthe claims, that the disclosure may be at least in part |ll implemented by means of both hardware and software, and that severa means" or "units" may be represented by the same item of hardware.

Although the figures may show a specific order of method steps, the order of the steps may differ from what is depicted. ln addition, two or more steps may be performed concurrently or with partial concurrence. Such variation will depend on the software and hardware systems chosen and on designer choice. All such variations are within the scope ofthe disclosure. Likewise, software implementations could be accomplished with standard programming techniques with rule-based logic and other logic to accomplish the various connection steps, processing steps, comparison steps and decision steps. The above mentioned and described embodiments are only given as examples and should not be limiting to the present disclosure. Other solutions, uses, objectives, and functions within the scope of the disclosure as claimed in the below described patent embodiments should be apparent for the person skilled in the art.

Claims

1. A method for assessing a printing process performed by a 3D bioprinter, the method comprising: obtaining sensor data associated with a plurality of sensor devices configured to obtain sensor detections of the printing process, the sensor data comprising sensor data associated with each sensor device of the plurality of sensor devices; processing the obtained sensor data, wherein the processing comprises any one of: analyzing each sensor data associated with each sensor device separately in order to create at least one separately analyzed sensor data point, and fusing the obtained sensor data of at least two sensor devices of the plurality of sensor devices in order to create one or more fused sensor data point(s) and analyzing each of the created one or more fused sensor data point(s) in order to create one or more analyzed-fused data point(s), and analyzing each sensor data associated with each sensor device separately in order to create at least one separately analyzed sensor data point and fusing the at least one separately analyzed sensor data point with at least one other separately analyzed sensor data point in order to create one or more fused-analyzed sensor data combination(s); and fusing the at least one separately analyzed sensor data point with at least one ofthe one or more analyzed-fused data point(s); determining a weighted component for each of: the at least one separately analyzed sensor data point, and/or the one or more analyzed-fused data point(s), and/or the one or more fused-analyzed sensor data combination(s), and/or 21 the fused at least one separately analyzed sensor data point with the at least one of the one or more analyzed-fused data point(s) by means of an association weight function in order to create one or more weighted processed data set(s); determining a sum vector based on the one or more weighted processed data set(s) by summing the one or more weighted processed data set(s); determining a state of the printing process by evaluating the determined sum vector against a print-process threshold value; and generating a state-signal indicative of the state of the printing process based on the evaluation of the sum vector against the print-process threshold value.

2. The method according to c|aim 1, wherein determining the state ofthe printing process comprises determining if the determined sum vector fulfills at least one requirement of the printing process, wherein the print-process threshold value comprises a predetermined threshold value for each ofthe at least one requirement of the printing process.

3. The method according to any one of claims 1 or 2, wherein the method further comprises: controlling the printing process based on the generated state-signal.

4. The method according to any one of claims 1 - 3, wherein the printing process comprises at least one of a pre-print phase, run-time print phase, and a post-print phase.

5. The method according to c|aim 4, wherein the method further comprising: controlling the printing process by forming a continuation or interruption decision for each one of the phases of the printing process based on the determined state ofthe printing process.

6. The method according to any one of claims 2 - 5, wherein the at least one requirement of the printing process comprises any one of an allowable calibration of a nozzle of the 3D bioprinter, an allowable priming ofthe nozzle ofthe 3D bioprinter, an allowable state of a dispensing needle connected to the nozzle of the 3D bioprinter, an indicator of clogging of the nozzle ofthe 3D printer, an allowable point tip calibration ofthe nozzle ofthe 3D bioprinter, an allowable flowrate and/or a volume, and/or a temperature, and/or a viscosity of a printing 22 fluid being introduced into the nozzle of the 3D bioprinter, and an allowable size and/or a volume of a hanging droplet ofthe printing fluid extruded out of the tip ofthe nozzle ofthe 3D bioprinter and/or at least one quality-of-print metric.

7. The method according to any of the preceding claims, wherein the plurality of sensor devices comprise any one of a camera, a thermal camera, a thermal sensor, a lidar, a radar, a laser, a Wheatstone bridge circuit arranged under a print bed of the 3D bioprinter, a pressure sensor, an ultrasound sensor, and a microwave radio frequency, RF, sensor.

8. The method according to any of the preceding claims, wherein the method further comprises: determining the weighted component, by means of the association weight function, by mu|tip|ying a predetermined weight value with each of the: at least one separately analyzed sensor data point, and/or the one or more analyzed-fused data point(s), and/or the one or more fused-analyzed sensor data combination(s), and/or the fused at least one separately analyzed sensor data point with the at least one of the one or more analyzed-fused data point(s).

9. A computer program carrier carrying one or more computer programs configured to be executed by one or more processors of a processing circuitry, the one or more programs comprising instructions for performing the method according to any one of claims 1 - 8, and wherein the computer program carrier is one of an electronic signal, optical signal, radio signal or a computer-readable storage medium.

10. A computer program product comprising instructions which, when the program is executed by one or more processors of a processing circuitry, causes the processing circuitry to carry out the method according to any one of claims 1 -

11. A system for assessing a printing process by a 3D bioprinter, the system comprising processing circuitry configured to: 23 obtain sensor data associated with a plurality of sensor devices configured to obtain sensor detections of the printing process, the sensor data comprising sensor data associated with each sensor device of the plurality of sensor devices; process the obtained sensor data, wherein the processing circuitry is configured to: analyze each sensor data associated with each sensor device separately in order to create at least one separately analyzed sensor data point, and fuse the obtained sensor data of at least two sensor devices of the plurality of sensor devices in order to create one or more fused sensor data point(s) and analyze each of the created one or more fused sensor data point(s) in order to create one or more analyzed-fused data point(s), and analyze each sensor data associated with each sensor device separately in order to create at least one separately analyzed sensor data point and fuse the at least one separately analyzed sensor data point with at least one other separately analyzed sensor data point in order to create one or more fused-analyzed sensor data combination(s); and fuse the at least one separately analyzed sensor data point with at least one of the one or more analyzed-fused data point(s); wherein the processing circuitry is further configured to: determine a weighted component for each of the at least one separately analyzed sensor data point, and/or the one or more analyzed-fused data point(s), and/or the one or more fused-analyzed sensor data combination(s), and/or the fused at least one separately analyzed sensor data point with the at least one of the one or more analyzed-fused data point(s) by means of an association weight function in order to create one or more weighted processed data set(s); determine a sum vector based on the one or more weighted processed data set(s) by summing the one or more weighted processed data set(s); determine a state of the printing process by evaluating the determined sum vector against a print-process threshold value; and 24 generate a state-signal indicative of the state of the printing process based on the evaluation of the sum vector against the print-process threshold value.

12. The system according to claim 11, wherein the processing circuitry is further configured to: determine the state of the printing process by determining if the determined sum vector fulfills at least one requirement of the printing process, wherein the print-process threshold value comprises a predetermined threshold value for each of the at least one requirement of the printing process.

13. The system according to any one of the claims 11 - 12, wherein the processing circuitry is further configured to: control the printing process based on the generated state-signal.

14. The system according to any one of the claims 11 - 13, wherein the printing process comprises at least one of a pre-print phase, run-time print phase, and a post-print phase.

15. The system according to claim 14, wherein the processing circuitry is further configured to: control the printing process by forming a continuation or interruption decision for each one of the phases of the printing process based on the determined state ofthe printing process.

16. The system according to any one of claims 12 - 15, wherein the at least one requirement of the printing process comprises any one of an allowable calibration of a nozzle of the 3D bioprinter, an allowable priming ofthe nozzle ofthe 3D bioprinter, an allowable state of a dispensing needle connected to the nozzle ofthe 3D bioprinter, an indicator of clogging of the nozzle ofthe 3D printer, an allowable point tip calibration ofthe nozzle ofthe 3D bioprinter, an allowable flowrate and/or a volume, and/or a temperature, and/or a viscosity of a printing fluid being introduced into the nozzle of the 3D bioprinter, and an allowable size and/or a volume of a hanging droplet ofthe printing fluid extruded out ofthe tip ofthe nozzle ofthe 3D bioprinter and/or at least one quality-of-print metric.

17. The system according to any one the claims 11 - 16, wherein the plurality of sensor devices comprise any one of a camera, a thermal camera, a thermal sensor, a lidar, a radar, a laser, a Wheatstone bridge circuit arranged under a print bed of the 3D bioprinter, a pressure sensor, an ultrasound sensor, and a microwave radio frequency, RF, sensor.

18. The system according to any one the claims 11 - 17, wherein the processing circuitry is further configured to: determine the weighted component, by means of the association weight function, by multiplying a predetermined weight value with each of: the at least one separately analyzed sensor data point, and/or the one or more analyzed-fused data point(s), and/or the one or more fused-analyzed sensor data combination(s), and/or the fused at least one separately analyzed sensor data point with the at least one of the one or more analyzed-fused data point(s).

19. A 3D bioprinter comprising: one or more print heads, each print head comprising a dispensing nozzle having an inlet for introduction of a printing fluid into the dispensing nozzle and an outlet for extruding the printing fluid from the outlet ofthe dispensing nozzle; and a system according to any one of claims 11 - 18 for assessing the printing process by the 3D bioprinter.