1. Home
  2. Vol 3, No 1 (2022)
  3. Akhtar

The Use of a Convolutional Neural Network in Detecting Soldering Faults from a Printed Circuit Board Assembly

Muhammad Bilal Akhtar

Abstract


Automatic Optical Inspection (AOI) is any method of detecting defects during a Printed Circuit Board (PCB) manufacturing process. Early AOI methods were based on classic image processing algorithms using a reference PCB. The traditional methods require very complex and inflexible preprocessing stages. With recent advances in the field of deep learning, especially Convolutional Neural Networks (CNN), automating various computer vision tasks has been established. Limited research has been carried out in the past on using CNN for AOI. The present systems are inflexible and require a lot of preprocessing steps or a complex illumination system to improve the accuracy. This paper studies the effectiveness of using CNN to detect soldering bridge faults in a PCB assembly. The paper presents a method for designing an optimized CNN architecture to detect soldering faults in a PCBA. The proposed CNN architecture is compared with the state-of-the-art object detection architecture, namely YOLO, with respect to detection accuracy, processing time, and memory requirement. The results of our experiments show that the proposed CNN architecture has a 3.0% better average precision, has 50% less number of parameters and infers in half the time as YOLO. The experimental results prove the effectiveness of using CNN in AOI by using images of a PCB assembly without any reference image, any complex preprocessing stage, or a complex illumination system.

 

Doi: 10.28991/HIJ-2022-03-01-01

Full Text: PDF


Keywords


Automatic Optical Inspection; Deep Learning; Neural Network; Object Detection; Printed Circuit Board Assembly; YOLO.

References


Huang, S. H., & Pan, Y. C. (2015). Automated visual inspection in the semiconductor industry: A survey. Computers in Industry, 66, 1–10. doi:10.1016/j.compind.2014.10.006.

Richter, J., Streitferdt, D., & Rozova, E. (2017). On the development of intelligent optical inspections. 2017 IEEE 7th Annual Computing and Communication Workshop and Conference, CCWC 2017, 1-6. doi:10.1109/CCWC.2017.7868455.

Day, R. H. (1972). Visual spatial illusions: A general explanation. Science, 175(4028), 1335–1340. doi:10.1126/science.175.4028.1335.

Coren, S., & Girgus, J. S. (1973). Visual spatial illusions: Many explanations. Science, 179(4072), 503–504. doi:10.1126/science.179.4072.503.

Moganti, M., Ercal, F., Dagli, C. H., & Tsunekawa, S. (1996). Automatic PCB inspection algorithms: A survey. Computer Vision and Image Understanding, 63(2), 287–313. doi:10.1006/cviu.1996.0020.

Janczki, M., Becker, kos, Jakab, L., Grf, R., & Takcs, T. (2013). Automatic Optical Inspection of Soldering. Materials Science - Advanced Topics. doi:10.5772/51699.

Fukushima, K. (1980). Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36(4), 193–202. doi:10.1007/BF00344251.

Raina, R., Madhavan, A., & Ng, A. Y. (2009). Large-scale deep unsupervised learning using graphics processors. Proceedings of the 26th International Conference on Machine Learning, ICML 2009, 873–880.

Taha, E. M., Emary, E., & Moustafa, K. (2014). Automatic Optical Inspection for PCB Manufacturing: a Survey. International Journal of Scientific & Engineering Research, 5(7), 1095–1102.

Harlow, C. A. (1982). Automated Visual Inspection: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-4(6), 557–573. doi:10.1109/TPAMI.1982.4767309.

Chin, R. T. (1988). Automated visual inspection: 1981 to 1987. Computer Vision, Graphics and Image Processing, 41(3), 346–381. doi:10.1016/0734-189X(88)90108-9.

Loh, H. H., & Lu, M. S. (1999). Printed circuit board inspection using image analysis. IEEE Transactions on Industry Applications, 35(2), 426–432. doi:10.1109/28.753638.

Pal Singh Chauhan, A., & Bhardwaj, S. C. (2011). Detection of bare PCB defects by image subtraction method using Machine Vision. Proceedings of the World Congress on Engineering 2011, WCE 2011, 2, 1597–1601.

Ozturk, S. (2017). Detection of PCB Soldering Defects using Template Based Image Processing Method. International Journal of Intelligent Systems and Applications in Engineering, 4(5), 269–273. doi:10.18201/ijisae.2017534388.

Lin, S. C., Chou, C. H., & Su, C. H. (2007). A development of visual inspection system for surface mounted devices on printed circuit board. IECON Proceedings (Industrial Electronics Conference), 2440–2445. doi:10.1109/IECON.2007.4459975.

Lee, D. T. (1978). A computerized cutomatic inspection system for complex printed thick film patterns. Technical Symposium East, 3, 172–177.

Hara, Y., Akiyama, N., & Karasaki, K. (1983). Automatic Inspection System for Printed Circuit Boards. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-5(6), 623–630. doi:10.1109/TPAMI.1983.4767453.

Hong, J. joong, Park, K. ja, & Kim, K. gu. (1998). Parallel processing machine vision system for bare PCB inspection. IECON Proceedings (Industrial Electronics Conference), 3, 1346–1350. doi:10.1109/iecon.1998.722846.

Sun, Y. N., & Tsai, C. T. (1992). A new model-based approach for industrial visual inspection. Pattern Recognition, 25(11), 1327–1336. doi:10.1016/0031-3203(92)90145-9.

Mandeville, J. R. (1985). Novel Method for Analysis of Printed Circuit Images. IBM Journal of Research and Development, 29(1), 73–86. doi:10.1147/rd.291.0073.

Belbachir, A. N., Lera, M., Fanni, A., & Montisci, A. (2005). An automatic optical inspection system for the diagnosis of printed circuits based on neural networks. Conference Record - IAS Annual Meeting (IEEE Industry Applications Society), 1, 680–684. doi:10.1109/IAS.2005.1518381.

Kobayashi, S., Tanimura, Y., & Yotsuya, T. (1990). Identifying solder surface orientation from color highlight images. IECON Proceedings (Industrial Electronics Conference), 1, 821–825. doi:10.1109/iecon.1990.149246.

Klein Wassink, R. J. (1984). Soldering in Electronics. In Soldering in Electron. Electrochemical Pub. doi:10.1016/0026-2714(90)90700-w.

Metzner, M., Fiebag, D., Mayr, A., & Franke, J. (2019). Automated optical inspection of soldering connections in power electronics production using convolutional neural networks. 2019 9th International Electric Drives Production Conference, EDPC 2019 - Proceedings. doi:10.1109/EDPC48408.2019.9011820.

Huang, W., & Wei, P. (2019). A PCB Dataset for Defects Detection and Classification. ArXiv, 1901 08204. Available online: http://arxiv.org/abs/1901.08204 (accessed on May 2021).

Tang, S., He, F., Huang, X., & Yang, J. (2019). Online PCB Defect Detector on a New PCB Defect Dataset. ArXiv. Available online: http://arxiv.org/abs/1902.06197 (accessed on May 2021).

Wu, F., & Zhang, X. (2014). An inspection and classification method for chip solder joints using color grads and Boolean rules. Robotics and Computer-Integrated Manufacturing, 30(5), 517–526. doi:10.1016/j.rcim.2014.03.003.

Fupei Wu, & Xianmin Zhang. (2011). Feature-Extraction-Based Inspection Algorithm for IC Solder Joints. IEEE Transactions on Components, Packaging and Manufacturing Technology, 1(5), 689–694. doi:10.1109/tcpmt.2011.2118208

Acciani, G., Brunetti, G., & Fornarelli, G. (2006). Application of neural networks in optical inspection and classification of solder joints in surface mount technology. IEEE Transactions on Industrial Informatics, 2(3), 200–209. doi:10.1109/TII.2006.877265.

Fanni, A., Lera, M., Marongiu, E., & Montisci, A. (2001). Neural Network Diagnosis for Visual Inspection in Printed Circuit Boards. 11th International Workshop on Principles of Diagnosis, 47–54.

Ko, K. W., & Cho, H. S. (2000). Solder joints inspection using a neural network and fuzzy rule-based classification method. IEEE Transactions on Electronics Packaging Manufacturing, 23(2), 78. doi:10.1109/6104.846932.

Xie, H., Zhang, X., Kuang, Y., & Ouyang, G. (2011). Solder joint inspection method for chip component using improved adaboost and decision tree. IEEE Transactions on Components, Packaging and Manufacturing Technology, 1(12), 2018–2027. doi:10.1109/TCPMT.2011.2168531.

Wu, H., Zhang, X., Xie, H., Kuang, Y., & Ouyang, G. (2013). Classification of solder joint using feature selection based on bayes and support vector machine. IEEE Transactions on Components, Packaging and Manufacturing Technology, 3(3), 516–522. doi:10.1109/TCPMT.2012.2231902.

Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90. doi:10.1145/3065386.

Sze, V., Chen, Y. H., Yang, T. J., & Emer, J. S. (2017). Efficient Processing of Deep Neural Networks: A Tutorial and Survey. Proceedings of the IEEE, 105(12), 2295–2329. doi:10.1109/JPROC.2017.2761740.

Zhao, Z. Q., Zheng, P., Xu, S. T., & Wu, X. (2019). Object Detection with Deep Learning: A Review. IEEE Transactions on Neural Networks and Learning Systems, 30(11), 3212–3232. doi:10.1109/TNNLS.2018.2876865.

Agarwal, S., Terrail, J. O. Du, & Jurie, F. (2018). Recent Advances in Object Detection in the Age of Deep Convolutional Neural Networks. ArXiv. Available online: https://arxiv.org/abs/1809.03193v2. (accessed on May 2021).

Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 580–587. doi:10.1109/CVPR.2014.81.

Uijlings, J. R. R., Van De Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013). Selective search for object recognition. International Journal of Computer Vision, 104(2), 154–171. doi:10.1007/s11263-013-0620-5.

Girshick, R., (2015). Fast R-CNN. ArXiv. Available online: https://arxiv.org/abs/1504.08083 (accessed on May 2021).

Ren, S., He, K., Girshick, R., Sun, J., & Faster, R. (2019). Towards real-time object detection with region proposal networks, 2016. ArXiv. Available online: https://arxiv.org/abs/1506.01497 (accessed on June 2021).

Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2014). Overfeat: Integrated recognition, localization and detection using convolutional networks. 2nd International Conference on Learning Representations, ICLR 2014 - Banff, AB, Canada, 1312-6229.

Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-December, 779–788. doi:10.1109/CVPR.2016.91.

Redmon, J., Farhadi, A., & YOLO9000. (2017). Better, Faster, Stronger. IEEE Conference on Computer Vision and Pattern Recognition CVPR, Honolulu, HI, USA, 517–6525.

Redmon, J., & Farhadi, A. (2018). YOLOv3: An Incremental Improvement. ArXiv. Available online: http://arxiv.org/abs/1804.02767 (accessed on May 2021).

Lin, Y. L., Chiang, Y. M., & Hsu, H. C. (2018). Capacitor Detection in PCB Using YOLO Algorithm. International Conference on System Science and Engineering, ICSSE 2018. doi:10.1109/ICSSE.2018.8520170.

Adibhatla, V. A., Chih, H.-C., Hsu, C.-C., Cheng, J., Abbod, M. F., & Shieh, J.-S. (2020). Defect Detection in Printed Circuit Boards Using You-Only-Look-Once Convolutional Neural Networks. Electronics, 9(9), 1547. doi:10.3390/electronics9091547.

Khare, T., Bahel, V., & Phadke, A. C. (2020). PCB-Fire: Automated Classification and Fault Detection in PCB. MPCIT 2020 - Proceedings: IEEE 3rd International Conference on “Multimedia Processing, Communication and Information Technology,” 123–128. doi:10.1109/MPCIT51588.2020.9350324.

Pramerdorfer, C., & Kampel, M. (2015). A dataset for computer-vision-based PCB analysis. Proceedings of the 14th IAPR International Conference on Machine Vision Applications, Tokyo, Japan, 378–381. doi:10.1109/MVA.2015.7153209.

Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. 3rd International Conference on Learning Representations, ICLR 2015 - San Diego, CA, USA.

Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, Massachusetts, USA, 1-9.

Lin, M., Chen, Q., & Yan, S. (2013). Network in network. ArXiv. Available online: https://arxiv.org/abs/1312.4400 (accessed on December 2021).

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-December, Cancun, Mexico, 770–778. doi:10.1109/CVPR.2016.90.

Luo, P., Wang, X., Shao, W., & Peng, Z. (2018). Towards understanding regularization in batch normalization. ArXiv. Available online: https://arxiv.org/abs/1809.00846 (accessed on December 2021).

Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15, 1929–1958.


Full Text: PDF

DOI: 10.28991/HIJ-2022-03-01-01

Refbacks

  • There are currently no refbacks.


Copyright (c) 2022 Muhammad Bilal Akhtar