Smart Access: Integrating Facial and Voice Biometrics with AI-Driven Deepfake and Spoofing Mitigation

Document Type : Original Article

Authors

School of Computing & Digital Tech, Eslsca University, Cairo, Egypt

Abstract

Smart Access (SA) is a modern, contactless access control system powered by artificial intelligence, designed to provide secure entry for spaces like offices, hospitals, hotels, and research facilities. Unlike traditional systems that rely on keys, PIN codes, RFID cards, or costly biometric devices, SA takes a more efficient and user-friendly approach. It uses multimodal biometric verification directly from a user's smartphone, removing the need for additional hardware.The system combines both facial and voice recognition with advanced deepfake detection to enhance security. Facial authentication is built on the DeepFace framework with a VGG-Face model, enhanced by liveness detection to block spoofing attempts. Voice recognition includes speaker verification through SpeechBrain, transcript checking with Whisper ASR, and deepfake voice detection using a fine-tuned Wav2Vec2 model. These features work together to defend against threats like replay attacks and AI-generated audio impersonations.
SA’s architecture includes a mobile or web client, a secure AI-powered backend, and an ESP32 microcontroller that controls physical access. When a user's identity is successfully verified, a secure signal is sent to the ESP32 to unlock the door. Administrators can manage users, permissions, rooms, and access records through an intuitive dashboard that supports multiple organizations with strict data separation. Performance evaluations showed impressive results: 97.4% accuracy in facial recognition, 94.6% in detecting fake audio, and an average verification time of just 2.4 seconds. In a user survey, over 90% of participants rated the system as more secure and convenient than traditional access methods.

Keywords

References

References

[1]      S. Lu, Z. Gao, Q. Xu, C. Jiang, A. Zhang and X. Wang, "Class-Imbalance Privacy-Preserving Federated Learning for Decentralized Fault Diagnosis With Biometric Authentication," in IEEE Transactions on Industrial Informatics, vol. 18, no. 12, pp. 9101-9111, Dec. 2022, doi: 10.1109/TII.2022.3190034.
[2]      H. Xing, S. Y. Tan, F. Qamar, and Y. Jiao, "Face anti-spoofing based on deep learning: A comprehensive survey," Applied Sciences, vol. 15, no. 12, p. 6891, 2025. doi: 10.3390/app15126891.
[3]      A. K. Jain, A. Ross, and S. Prabhakar, “An introduction to biometric recognition,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 14, no. 1, pp. 4–20, Jan. 2004.
[4]      U.S. Department of Homeland Security, “2024 Update on DHS’s Use of Face Recognition & Face Capture Technologies,” Jan. 16, 2025. [Online]. Available: www.dhs.gov
[5]      A. Hadid, N. Evans, S. Marcel, and J. Fierrez, “Biometrics systems under spoofing attack: An evaluation methodology and lessons learned,” IEEE Signal Processing Magazine, vol. 32, no. 5, pp. 20–30, Sep. 2015.
[6]      D. A. Reynolds, T. F. Quatieri, and R. B. Dunn, “Speaker verification using adapted Gaussian mixture models,” Digital Signal Processing, vol. 10, no. 1–3, pp. 19–41, Jan. 2000.
[7]       Cyberlink, “What is Facial Recognition? - The 2025 Ultimate Guide to Facial Recognition Technology,” Dec. 10, 2024. [Online]. Available: www.cyberlink.com
[8]      R. Brunelli and D. Falavigna, “Person identification using multiple cues,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 10, pp. 955–966, Oct. 1995.
[9]      F. Schroff, D. Kalenichenko, and J. Philbin, “FaceNet: A unified embedding for face recognition and clustering,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Boston, MA, USA, Jun. 2015, pp. 815–823.
[10]   ABA Banking Journal, “Challenges in voice biometrics: Vulnerabilities in the age of deepfakes,” Feb. 15, 2024. [Online]. Available: bankingjournal.aba.com
[11]   Bioconnect, “The Future of Facial Authentication & Biometrics: 8 Emerging Trends to Watch,” Apr. 18, 2024. [Online]. Available: bioconnect.com
[12]   Springer, “Advancements in detecting Deepfakes: AI algorithms and future prospects − a review,” May 7, 2025. [Online]. Available: link.springer.com
[13]   Aware, “How to Offer Powerful Defense Against Deepfakes with Biometrics,” Apr. 26, 2024. [Online]. Available: www.aware.com
[14]   PMC, “Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead,” 2025. [Online]. Available: pmc.ncbi.nlm.nih.gov
[15]   Security Boulevard, “Deepfake Detection – Protecting Identity Systems from AI-Generated Fraud,” Feb. 3, 2025. [Online]. Available: securityboulevard.com
[16]   A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” in Advances in Neural Information Processing Systems, Dec. 2020, pp. 12449–12460.
[17]   ISACA, “White Papers 2024 Examining Authentication in the Deepfake Era,” Jul. 29, 2024. [Online]. Available: www.isaca.org
[18]   iProov, “How Deepfakes Threaten Remote Identity Verification Systems,” Jan. 11, 2024. [Online]. Available: www.iproov.com

Files

Share

How to cite

Statistics