Enhancing CNC Machine Operator Accessibility through a Multimodal Chatbot


Authors : Harsh Sanchaniya; Dhruv Sinha; Ashish Joshi; Sneha Kothimbire; Dr. Bharati Vasgi; Punam Chavan

Volume/Issue : Volume 10 - 2025, Issue 4 - April


Google Scholar : https://tinyurl.com/3ezzrmrc

Scribd : https://tinyurl.com/5d9e83ws

DOI : https://doi.org/10.38124/ijisrt/25apr1796

Google Scholar

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Note : Google Scholar may take 15 to 20 days to display the article.


Abstract : Modern CNC machining presents significant operational complexities and data interaction challenges, often creating accessibility barriers for a diverse operator workforce. This paper details the design, development, and accessibility-focused evaluation of a Flutter-based mobile conversational assistant tailored for CNC machine operators. Developed with industry collaboration, the system aims to bridge the accessibility gap by translating complex, real-time telemetry data (spindle speed, feed rate, alarms) into easily understandable, actionable insights. The architecture leverages IoT data streams, structured storage, efficient querying, and automated data processing. Crucially, it employs a multimodal interface (text and voice), multilingual support, and a conversational interaction model powered by a Large Language Model (LLM) with Retrieval-Augmented Generation (RAG). Specific features like hands-free continuous conversation mode and visual adjustments directly target physical, cognitive, and linguistic accessibility needs. By providing intuitive, context-aware guidance through natural language, the assistant empowers operators with varying technical literacy and language backgrounds, reduces cognitive load, facilitates hands-free information access, and aims to foster a more inclusive and efficient shop floor environment. Initial findings suggest significant potential in reducing task completion times and improving usability compared to traditional interfaces.

Keywords : CNC Machines, Operator Accessibility, Human-Computer Interaction, Conversational AI, Chatbot, Multimodal Interface, Multilingual Support, Industry 4.0, Flutter, Inclusive Design

References :

  1. F. Tao and M. Zhang, "Digital Twin Shop-Floor: A New Shop-Floor Paradigm Towards Smart Manufacturing," IEEE Access, vol. 5, pp. 22780-22790, Aug. 2017, doi: 10.1109/ACCESS.2017.2756069
  2. C. Yang, S. Lan, L. Wang, W. Shen, and G. G. Q. Huang, "Big Data Driven Edge-Cloud Collaboration Architecture for Cloud Manufacturing: A Software Defined Perspective," IEEE Access, vol. 8, pp. 41947-41958, Mar. 2020, doi: 10.1109/ACCESS.2020.2977846.
  3. Q. Qi and F. Tao, "A Smart Manufacturing Service System Based on Edge Computing, Fog Computing, and Cloud Computing," IEEE Access, vol. 7, pp. 84553-84562, Jun. 2019, doi: 10.1109/ACCESS.2019.2923610.
  4. Y. Xu, F. Tao, D. Cheng, and L. Liu, "A Cyber-Physical System Architecture for Smart Manufacturing," IEEE Transactions on Industrial Informatics, vol. 12, no. 4, pp. 1415-1423, Apr. 2016, doi: 10.1109/TII.2015.2467587.
  5. F. Tao, Z. Cheng, and X. Zhang, "Digital Twin Driven Smart Manufacturing," Procedia CIRP, vol. 72, pp. 57-62, 2018, doi: 10.1016/j.procir.2018.03.115.
  6. M. M. Goh, H. W. Tan, and Y. S. Lee, "Cloud-based Manufacturing: A New Manufacturing Model," Journal of Manufacturing Science and Engineering, vol. 137, no. 2, pp. 021015- 021022, Mar. 2015, doi: 10.1115/1.4029766.
  7. Y. Huang, Y. Song, Z. Xu, and J. Liu, "Cloud Computing and Big Data Analytics for Smart Manufacturing: A Review," Journal of Manufacturing Processes, vol. 38, pp. 83-99, Oct. 2019, doi: 10.1016/j.jmapro.2019.01.015.
  8. M. Penica, M. Bhattacharya, W. O’Brien, S. McGrath, M. Hayes and E. O’Connell, "Adaptable Decision Making Chatbot System: Unlocking Interoperability in Smart Manufacturing," 2023 International Conference on Computing, Electronics & Communications Engineering (iCCECE), Swansea, United Kingdom, 2023, pp. 23-29, doi: 10.1109/iCCECE59400.2023.10238531.
  9. Z. Yuan, H. Ding, M. Li, L. Li and G. Q. Huang, "AiFashion: Multi-Modal and Multi-Dimensional Large Model Based on Self-Trained Customer Digital-Twin for Fashion Design and Manufacturing," 2024 International Conference on Automation in Manufacturing, Transportation and Logistics (ICaMaL), Hong Kong, 2024, pp. 1-6, doi: 10.1109/ICaMaL62577.2024.10919567.
  10. O. Ahaneku, M. Siegl, S. Stromberger and R. Vidrascu, "A Scalable AI-Driven Chatbot for Real-Time Diagnostics in Manufacturing Plants: Merging Google Dialogflow, BERT, and a Self-Learning Module," 2023 IEEE 12th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), Dortmund, Germany, 2023, pp. 853-858, doi: 10.1109/IDAACS58523.2023.10348738.
  11. L. Wang, Y. Lu, and H. Shen, "Cloud-based Cyber-Physical Systems for Smart Manufacturing," Procedia CIRP, vol. 60, pp. 103-108, 2017, doi: 10.1016/j.procir.2017.01.029.
  12. M. Liu, G. Sun, and Y. Huang, "Cloud Manufacturing: A New Manufacturing Model," Journal of Cloud Computing: Advances, Systems and Applications, vol. 3, no. 1, pp. 3-11, Dec. 2014, doi: 10.1186/s13677-014-0022-4.
  13. L. Zhang, W. Zhong, J. Liu, and W. Zhou, "Smart Manufacturing Based on Cloud Computing and Internet of Things," International Journal of Advanced Manufacturing Technology, vol. 101, no. 9-12, pp. 3127-3137, Jan. 2019, doi: 10.1007/s00170-019-03273-5.
  14. Y. Chen, Y. Gao, and Q. Liang, "Smart Manufacturing for the Internet of Things," Journal of Intelligent Manufacturing, vol. 28, no. 7, pp. 1581-1595, Dec. 2017, doi: 10.1007/s10845- 015-1019-1.
  15. D. Mourtzis, A. Vlachou, and V. Zogopoulos, ‘‘Cloud-based augmented reality remote maintenance through shop-floor monitoring: A product-service system approach,’’ ASME J. Manuf. Sci. Eng., vol. 139, no. 6, pp. 152–157, Jan. 2017.
  16. N. Gkatzios et al., "A Chatbot Assistant for Optimizing the Fault Detection and Diagnostics of Industry 4.0 Equipment in the 6G era," 2023 IEEE Conference on Standards for Communications and Networking (CSCN), Munich, Germany, 2023, pp. 124-129, doi: 10.1109/CSCN60443.2023.10453129.
  17. T. P. Nagarhalli, V. Vaze and N. K. Rana, "A Review of Current Trends in the Development of Chatbot Systems," 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 2020, pp. 706-710, doi: 10.1109/ICACCS48705.2020.9074420.

Modern CNC machining presents significant operational complexities and data interaction challenges, often creating accessibility barriers for a diverse operator workforce. This paper details the design, development, and accessibility-focused evaluation of a Flutter-based mobile conversational assistant tailored for CNC machine operators. Developed with industry collaboration, the system aims to bridge the accessibility gap by translating complex, real-time telemetry data (spindle speed, feed rate, alarms) into easily understandable, actionable insights. The architecture leverages IoT data streams, structured storage, efficient querying, and automated data processing. Crucially, it employs a multimodal interface (text and voice), multilingual support, and a conversational interaction model powered by a Large Language Model (LLM) with Retrieval-Augmented Generation (RAG). Specific features like hands-free continuous conversation mode and visual adjustments directly target physical, cognitive, and linguistic accessibility needs. By providing intuitive, context-aware guidance through natural language, the assistant empowers operators with varying technical literacy and language backgrounds, reduces cognitive load, facilitates hands-free information access, and aims to foster a more inclusive and efficient shop floor environment. Initial findings suggest significant potential in reducing task completion times and improving usability compared to traditional interfaces.

Keywords : CNC Machines, Operator Accessibility, Human-Computer Interaction, Conversational AI, Chatbot, Multimodal Interface, Multilingual Support, Industry 4.0, Flutter, Inclusive Design

Never miss an update from Papermashup

Get notified about the latest tutorials and downloads.

Subscribe by Email

Get alerts directly into your inbox after each post and stay updated.
Subscribe
OR

Subscribe by RSS

Add our RSS to your feedreader to get regular updates from us.
Subscribe