Enhanced Outcomes in Intelligent Manufacturing: Reliability Assessment of an Automated Quality Inspection Report Generation Tool
Release time:
2026-08-26 17:31
Source:

Quality inspection covers the entire process, from incoming materials to work-in-progress and outgoing products. [1] , its deliverables are never “pass/fail,” but rather A report that can be signed, traced, and reused The empirical findings over the past year are highly consistent: a model can detect defects, but that does not mean it can explain them.
01 What are the key factors driving improved outcomes in smart manufacturing?
▍The key to success lies not in isolated intelligence, but in layered controllability.
The reference architecture for large models in intelligent manufacturing divides responsibilities into three layers: Semantic Reasoning Layer (explaining the context, breaking down the goals), Adaptive Cooperative Control Layer and Independent Assurance Execution Layer (Classic controllers, runtime monitoring, motion constraints), with the digital twin serving as a shared foundation. The key constraint is: semantic-layer output. It remains merely a “proposal” and must undergo grounding and verification. , its natural language rationale should not be assumed to be faithful [2] 。
▍Upstream machine readability determines the ceiling for downstream automation
The trustworthy starting point for report automation must therefore be deterministic engineering data. MBDConnect for Creo 、 MBDConnect for NX and FormatWorks This is precisely the step being taken: exporting semantic PMI from various CAD platforms into neutral formats such as QIF and STEP AP242, enabling downstream systems to receive input that is both parsable and traceable.
02 Pain points in preparing quality inspection reports
▍Testing and reporting have long been disconnected.
Industrial inspection requires both precise localization and structured reporting, yet in practice these are handled separately, leaving linguistic interpretation to human experts. [3] The detector outputs bounding boxes; the report requires failure modes, severity, and process procedures.
▍The cost of simply having general-purpose large models write reports is underestimated
In the wind turbine blade inspection benchmark, the zero-shot general-purpose vision-language model baseline achieved a BLEU-4 score of only 0.07 , hallucination rate 65% Expert scoring 3.3/10 One of the root causes is resolution loss: the visual encoder resamples the input to 336×336, which affects the recall of small defects by the dedicated detector on images in the same batch. 0.852 , GPT-4V only 0.22 [3] 。
▍The Fragility of “Unified Models” in Accuracy-Critical Scenarios
InspectVLM, a unified model trained on over 290,000 drone images, achieves an image-level classification accuracy of 75.9% (compared to 66.7% for ResNet-50), yet its fine-grained detection precision/recall is only 16.5/19.8 , far lower than Faster R-CNN's 46.1/43.7; when the prompt template is fixed, the model from the 4th training epoch onwards Answer “yes” to all tasks without exception. , effectively ignores the image input [4] 。
“Standardizing the prompt format—a common practice on production testing lines—may inadvertently create” Accurate on the validation set but fails when wording or operating conditions change slightly. “A fragile model.”
—— Wallace et al., InspectVLM [4]
03 Automatic Quality Inspection Report Generation Tool
▍Decoupled Pipeline: Separating “Read” and “Write” Operations
Detector localization, deterministic module encoding detection boxes into spatial tokens injected into the prompt, and domain-fine-tuned small model outputting structured JSON reports. Three-step ablation. [3] :
- Coordinate Injection (Zero Training Cost) : The spatial illusion rate is determined by 58% down to 14% 。
- QLoRA Domain Fine-Tuning : A 1.5-billion-parameter model fine-tuned on 947 domain-specific reports, in terms of BLEU-4 (0.36 vs. 0.19), hallucination rate (18% vs. 29%), and expert score (7.4 vs. 5.9) A general-purpose model with over 671 billion parameters across the board 。
- Retrieval-Augmented Fine-Tuning (RAFT) : The hallucination rate has dropped again to 4% , the process compliance rate increased from 41% to 89% ; Complete System Expert Score: 8.6/10.
▍Knowledge grounding, self-reflection, and multimodal compensation
The plug-and-play knowledge-grounded reasoning framework boosts the benchmark reasoning accuracy for multimodal anomalies to 69.90% (Retrieval accuracy 83%) [5] ; After modeling “reflection” as a learnable decision correction, the 4-billion-parameter model achieved a localization score of 0.653 , surpassing multiple commercial models [6] . A few-shot vision-language approach overcomes the bottleneck of AOI requiring retraining whenever new defects emerge: in None occurred during training. Obtained on the defect class 82.9% Accuracy, under the same conditions, ResNet-50 is 0.00% [7] ; The vision–tactile–language model can compensate for occlusions and reflections. [1] 。
▍Deterministic Generation: Direct Report Output from Semantic PMI
Complementing this is the deterministic route. MBDVidia Directly read semantic PMI from 3D models, generate a Bill of Characteristics (BoC), and output an AS9102 First Article Inspection Report with one click, supporting PDF/HTML/Excel/Net-Inspect; forms and model characteristics—tolerances are bidirectionally linked. Its content can be traced back item by item to model annotations, eliminating phantom issues; the prerequisite is precisely the machine readability discussed in Section 01—by MBDConnect for Creo/NX and FormatWorks CAD system plugins and MBDVidia Software Export QIF/STEP AP242.
04 Reliability of Automated Generation of Quality Inspection Reports
▍Strong initial screening does not necessarily mean the re-evaluation is reliable.
The lithography defect understanding benchmark decomposes review into four tasks: classification, morphology analysis, coarse localization, and cause analysis. On 1,761 real-world review images, the best commercial model achieves a macro F1 score of 0.932 , biometric recognition plummeted to 0.201 , the best cause analysis is only 0.216, and the closed-loop LCS score is 0.474 [8] . Benchmark for industrial anomaly detection: Commercial models achieve 81%–84% accuracy in texture-based scenarios, but this drops to 51%–68% for workpieces and electronic components; localization scores only 0.103–0.201 [6] ; In the PCB benchmark, GPT-5's F1 score is only 20.2 , less than half of the dedicated model 51.1 [9] 。
"This indicates that in PCB quality inspection scenarios, Strong language generation and semantic alignment capabilities do not automatically translate into reliable spatial localization abilities. 。”
—— Sun et al., UniPCB [9]
▍“Seeing the evidence” does not equal “correct attribution”
Most alarming is this diagnosis: in the analysis of causes for 250 manually scored cases, the mean evidence relevance 0.992 , hallucination rate as low as 1.2% , but the highest score for causation is still only 0.216 [8] 。 A low illusion rate does not equate to high reliability. —It is dangerous to use a single-metric acceptance tool based on hallucination rate, even if every statement in the report is supported by evidence but the conclusions are incorrect.
▍More context isn’t always better; the evaluation itself must be reproducible.
An adaptive RAG evaluation for semiconductor manufacturing (4K–32K window, 52,800 evaluations) found that at 4K, model performance differed by up to 45%; at 28K, results converged within 1% bandwidth; and beyond 28K, anomalies emerged. Attention dilution , relevance and support quality declined by 6.1% and 6.0%, respectively [10] . The credibility of the conclusions also depends on the evaluation protocol: the primary metrics of the lithography benchmark mentioned above are all generated by deterministic scorers, with high efficiency 99.88% , Agreement rate for blind labels: 79.0% (κ=0.727) [8] . At the governance level: there is both work in the manufacturing sector Not a single report describes formal safety verification performed prior to executing outputs from large models. [2] 。
Three Practical Reliability Guidelines
I. Source identification. The factual foundation is built upon semantic PMI and a neutral format, by MBDVidia 、 MBDConnect for Creo/NX and FormatWorks and other similar MBD data management tools ensure that the data is machine-readable.
II. Generate mandatory grounding. By combining coordinate injection, domain fine-tuning, and retrieval augmentation, the hallucination rate can be reduced from 65% to 4%. [3] , it does not hold without any one link.
III. Tiered Signing. Set thresholds based on semantic hierarchy: screening can be automated, but morphological analysis, localization, and conclusions regarding origin must retain manual review and an independent assurance layer. [2][8] 。
References
[1] Zong, J.; Jia, Q.; Shi, M.; Li, T.; Li, J.; Lv, Z.; Chen, G.; Deng, F. VitaTouch: Property-Aware Vision–Tactile–Language Model for Robotic Quality Inspection in Manufacturing. arXiv :2604.03322, 2026-04-02. (School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications, et al.)
[2] Bahrpeyma, F.; Reichelt, D. A MARL Centered Reference Architecture for Large Language Model Augmentation in Smart Manufacturing. arXiv :2608.07148, 2026-08-07. (Intelligent Production Systems, Dresden University of Applied Sciences, Germany)
[3] Malikussaid; Gohar, I. A Hybrid Vision-Language Architecture for Automated Defect Reasoning and Report Generation in Industrial Inspection. arXiv :2605.26533v1 [cs.CV], 2026-05-26. (School of Computing, Telkom University; School of Engineering and Technology, Sunway University, Malaysia)
[4] Wallace, C.; Corley, I.; Lwowski, J. InspectVLM: Unified in Theory, Unreliable in Practice. arXiv :2508.01921v1 [cs.CV], 2025-08-03. (Zeitview)
[5] Zhang, K.; Zhang, Z.; Sun, X.; Wang, A.; Nie, J.; Chen, Q.; Hao, H.; Guo, J.; Zhang, J. ADSeeker: A Knowledge-Grounded Reasoning Framework for Industry Anomaly Detection and Reasoning. arXiv :2508.03088v1, 2025-08-05. (Shandong University, City University of Hong Kong)
[6] Huang, C.; Li, Y.; Cao, Y.; Wang, W.; Huang, H.; Wen, J.; Ren, W.; Cao, X. M3-AD: Reflection-aware Multi-modal, Multi-category, and Multi-dimensional Benchmark and Framework for Industrial Anomaly Detection. arXiv :2603.00055v1 [cs.LG], 2026-02-10. (Shenzhen Campus of Sun Yat-sen University, Hunan University, Harbin Institute of Technology Shenzhen)
[7] Dinh, H. L.; Doan, D. V.; Nguyen, N. K.; Nguyen, D. T.; Hoang, H. H. Autonomous Defect Classification in Manufacturing: A Novel Few-Shot Vision-Language Modeling Approach. Engineering, Technology & Applied Science Research , 2026, 16(3): 36545–36551. DOI: 10.48084/etasr.18859. (Faculty of Control and Automation, Vietnam Electricity University; accepted on 2026-05-11, CC-BY 4.0 open access)
[8] Ji, H.; Zhao, B.; Lv, S.; Lv, Y. LDU-Bench: Multimodal LLM Evaluation for Lithography Defect Understanding under Layout-Varying Circuit Backgrounds. arXiv :2608.03078v1 [cs.CV], 2026-08-04.
[9] Sun, F.; Jiang, X.; Wu, J.; Zhang, H.; Zheng, F.; Yang, J. UniPCB: A Unified Vision-Language Benchmark for Open-Ended PCB Quality Inspection. arXiv :2601.19222, 2026-01-27. (Shenzhen Polytechnic University, Southern University of Science and Technology, Liaoning University of Science and Technology)
[10] Qian, J.; Yi, C.; Xia, M.; Wu, W.; Zhu, J.; Guan, J. FAB-Bench: A Framework for Adaptive RAG Benchmarking in Semiconductor Manufacturing. arXiv :2605.26476 (preprint), 2026-05-26. (FutureFab.AI)
Results of Intelligent Manufacturing,Automatic Generation of Quality Inspection Reports,Semantic PMI Propagation,MBDVidia,MBD plugin for CAD systems,PMI machine-readable