During this phase of the project, we focused on comparing two different approaches for the error detection subsystem of AutoPrint.
As explained in the previous posts, one option was based on Obico, a monitoring solution that uses AI to detect failures during 3D prints. In parallel, we also explored a more custom machine learning approach based on YOLOv11 nano, trained with a custom-filtered dataset focused on relevant 3D printing error classes.
The goal of this comparison was to understand which option would be more suitable for our system. Since AutoPrint aims to work with minimal human intervention, the error detection subsystem must be reliable, fast enough to react during a failed print and easy to integrate into the rest of the automatic workflow.
1. Why We Compared These Two Approaches
The main objective of this comparison was to evaluate two possible solutions for the same problem: detecting failures during a 3D print.
The Obico-based approach represents a more complete and ready-to-use solution. It already includes printer monitoring, camera-based error detection, notifications and the possibility of pausing a print when a problem is detected.
The YOLOv11 nano approach, on the other hand, represents a more custom solution. It gives us more control over the model, the dataset, the selected error classes and the future integration with our own system.
By testing both options, we wanted to understand the trade-off between:
- a solution that is already mature and easier to use;
- a solution that is more flexible and customizable;
- the speed of error detection;
- the reliability of the detections;
- the confidence level of the predictions;
- the effort needed to integrate each option into AutoPrint.
This comparison was important because the error detection subsystem directly affects the safety and autonomy of the entire project.
2. Test Cases Used for the Comparison
To make the comparison more relevant, we used the same type of test cases previously used to evaluate Obico.
One of the most important tests was the provoked error test. In this test, a specific part was designed to fail during printing. The part consisted of a cone with a wide cylinder on top, printed without supports. This geometry was intentionally chosen because the upper section would not have enough support, creating an unstable print and eventually causing loose filament to appear during the process.
This test was useful because it simulated a real failure scenario where the printer continues extruding material even though the printed object is no longer being formed correctly.
The same type of failure was then used to evaluate the YOLOv11 nano model after it had been trained with the custom-filtered dataset.
Besides this provoked error test, we also considered the importance of monitoring normal prints without generating unnecessary false alarms. In the Obico tests, this included a long print of around 18 hours, where no error was detected, which was the expected result.
3. Results with Obico
In the provoked error test, Obico was able to detect the failure very quickly after the problem started.
This was one of the strongest points observed during testing. The system identified the error before the accumulation of loose filament became too large, which is very important for an autonomous printing system.
Fast detection matters because, if the error is detected early, the system can react before the situation becomes worse. For example, the print can be paused, the user can be notified and the automatic sequence can be stopped before the system tries to remove a failed part or start a new print.
Obico also showed good behavior during normal prints. In the long print test, it monitored the print without reporting an error, which suggests that it can be used continuously without necessarily generating false alarms in successful prints.
Overall, Obico performed well as a monitoring and error detection solution.
4. Results with YOLOv11 Nano
The YOLOv11 nano model was also tested after being trained with the custom-filtered dataset.
This approach was useful because it allowed us to explore a more personalized solution, focused on the specific error classes selected for our project, such as spaghetti, stringing, warping and bed adhesion problems.
However, when compared with Obico in the provoked error test, the YOLOv11 nano model was not able to provide the same level of early and reliable detection.
In some cases, the model was able to detect possible errors relatively early in the failure process. However, these early detections usually had a low confidence level, often below 50%. This means that, although the model was already identifying that something could be wrong, the prediction was not strong enough to be safely used as an automatic decision trigger.
As the error became more visually obvious, the model’s detections became more reliable. In practice, this means that the print failure had to be more advanced before the YOLOv11 nano model could detect it with a confidence level that would be more useful for decision-making.
For an autonomous printing workflow, this is an important limitation. If the system reacts too early to a low-confidence detection, it may pause prints unnecessarily. However, if it waits until the confidence is high enough, the failure may already be more developed, increasing the risk of wasted material, failed automation steps and possible obstruction around the print area.
Even so, the YOLOv11 nano approach remains valuable from a technical point of view. It gave us a better understanding of how a custom model could be trained, tested and potentially improved in the future using images collected from our own setup.
5. Main Comparison Between the Two Solutions
After testing both approaches, the main difference we observed was the speed and reliability of detection.
Obico detected the provoked failure much earlier and with more practical reliability. This makes it more suitable for an autonomous system where quick reaction is important.
YOLOv11 nano, on the other hand, could sometimes identify possible errors relatively early, but those early detections usually had a low confidence level, often below 50%. Because of this, they would not be reliable enough to automatically pause a print or trigger a critical system decision.
To obtain more confident detections, the error generally needed to be more developed and visually obvious. This may be related to the dataset used for training, the type of images available, the selected classes or the fact that the model was not yet trained with enough images from our own printer setup.
The comparison can be summarized as follows:
- Obico detected the provoked error earlier and more reliably
- Obico already includes monitoring, alerts and printer integration features
- YOLOv11 nano sometimes detected possible errors early, but with low confidence
- YOLOv11 nano gives more control over the model and the dataset
- YOLOv11 nano can be improved over time with more representative data
- YOLOv11 nano requires more development and validation before being used as the main solution
- Obico is more ready to use in the current stage of the project
This does not mean that the YOLOv11 nano approach failed. Instead, it showed that a custom model is possible, but it would require more data, more tuning and more real-world testing before it could match the practical performance of Obico.
6. Final Decision
After comparing both solutions, we decided that Obico is currently the best option for the AutoPrint error detection subsystem.
The main reason for this decision was its faster and more reliable detection during the provoked error test. Since the system is designed to operate with minimal human supervision, detecting a failure as early as possible is one of the most important requirements.
Although YOLOv11 nano sometimes produced early detections, these were generally associated with low confidence values. In the current state of the model, this makes it difficult to use those early predictions as the basis for automatic actions.
Obico also has the advantage of already being integrated with the printer monitoring workflow. It can receive camera data, detect failures, notify the user and potentially pause the print. This makes it more practical for the current version of the prototype.
Although YOLOv11 nano offers more customization and remains an interesting option for future development, Obico provides a more reliable and complete solution at this stage of the project.
Conclusion
This comparison allowed us to evaluate two different approaches for 3D printing error detection in AutoPrint.
The YOLOv11 nano solution showed that it is possible to train a custom model using a filtered dataset focused on relevant printing errors. This approach gives us flexibility and could become stronger in the future if trained with more images from our own printer setup.
However, in the tests performed, Obico detected the provoked failure earlier and more reliably than the YOLOv11 nano model. While YOLOv11 nano was sometimes able to detect possible problems relatively early, these detections often had confidence levels below 50%, making them less suitable for automatic decision-making.
Since early and reliable detection is critical for preventing wasted material and stopping the automatic workflow before it continues incorrectly, this was a decisive factor.
For this reason, we decided to use Obico as the preferred solution for the error detection subsystem of AutoPrint.
The YOLOv11 nano work remains useful as a technical exploration and as a possible future improvement path, but for the current prototype, Obico offers the best balance between reliability, integration and practical performance.