Resilience and Redundancy in Enterprise Storage: A Comparative Study of Erasure Coding vs. Replication in Distributed File Systems

Paper ID: EIJTEM_2024_11_4_141-144

Author's Name: Lakshmidhar Kotipalli

Volume: 11

Issue: 4

Year: 2024

Page No: 141-144

Abstract:

As enterprises move toward highly distributed storage infrastructures, ensuring data durability and availability has become a top priority. This research compares the resilience, performance, and cost implications of erasure coding and traditional replication across systems like Hadoop Distributed File System (HDFS), Amazon S3, and Azure Blob Storage. Through fault injection and failure recovery simulations, the study evaluates recovery times, storage overhead, and network impact. The findings provide insights for IT decision-makers balancing performance with fault tolerance in large-scale data environments.

Keywords: Erasure Coding, Replication, Distributed File Systems, HDFS, Amazon S3, Azure Blob Storage, Resilience, Fault Tolerance, Data Durability, Enterprise Storage

References:

1. Amazon Web Services. (2021). Amazon S3 Storage Classes. https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html
2. Ghemawat, S., Gobioff, H., & Leung, S.-T. (2003). The Google File System. SOSP, 29–43.
3. Jena, J. (2015). Next-Gen Firewalls Enhancing: Protection against Modern Cyber Threats. International Journal of Multidisciplinary and Scientific Energing Research, 3(4), 2015-2019. https://ijmserh.com/admin/pdf/2015/10/46_Next.pdf
4. Huang, C., Simitci, H., Xu, Y., Ogus, A., Calder, B., Gopalan, P., ... & Yekhanin, S. (2012). Erasure coding in Windows Azure Storage. USENIX Annual Technical Conference, 15-26.
5. Talluri Durvasulu, M. B. (2022). Exploring the Power of Cloud Storage with Azure and AWS. International Journal on Recent and Innovation Trends in Computing and Communication, 10(2), 59-65. https://ijritcc.org/index.php/ijritcc/article/view/11423
6. Kavulya, S., Gandhi, R., & Narayanasamy, S. (2019). Resource trade-offs in erasure-coded distributed storage. Proceedings of the VLDB Endowment, 12(5), 563–574.
7. Gudimetla, S. R., & Kotha, N. R. (2024). AI-driven cybersecurity: Enhancing threat detection and response strategies. International Research Journal of Modernization in Engineering Technology and Science, 6(5), 1374–1376. https://doi.org/10.56726/IRJMETS55883
8. Li, J., Lee, K., & Li, D. (2015). Data reliability in cloud storage systems: A comparative study. Cluster Computing, 18(3), 1063–1076.
9. Munnangi, S. (2017). Composable BPM: Modularizing workflows for agility and efficiency. Turkish Journal of Computer and Mathematics Education, 8(2), 409–420. https://doi.org/10.61841/turcomat.v8i3.14973
10. Bellamkonda, S. (2018). Data security: Challenges, best practices, and future directions. International Journal of Communication Networks and Information Security, 10, 256–259. https://www.ijcnis.org/index.php/ijcnis/article/view/7526
11. Microsoft Azure. (2022). Redundancy options. https://learn.microsoft.com/en-us/azure/storage/common/storage-redundancy
12. Kolla, S. (2023). Green Data Practices: Sustainable Approaches to Data Management. International Journal of Innovative Research in Computer and Communication Engineering, 11(11), 11451-11457. https://doi.org/10.15680/IJIRCCE.2023.1111001
13. Rashmi, K. V., Shah, N. B., & Kumar, P. V. (2013). A hitchhiker's guide to fast and efficient data reconstruction in erasure-coded data centers. ACM SIGCOMM Computer Communication Review, 43(4), 331–342.
14. Wei, W., & Guo, Y. (2021). Hybrid data protection strategies for big data storage systems. Journal of Cloud Computing, 10(1), 19.
15. Vangavolu, S. V. (2022). Implementing microservices architecture with Node.js and Express in MEAN applications. International Journal of Advanced Research in Engineering and Technology, 13(8), 56–65. https://doi.org/10.34218/IJARET_13_08_007
16. Xie, L., Yang, D., & Chen, H. (2016). A cost-effective approach to optimize storage with erasure coding. IEEE Transactions on Computers, 65(4), 1136–1147.
17. Zhang, Z., Li, D., & Xu, C. (2020). Dynamic adaptation of erasure coding for cloud storage systems. IEEE Access, 8, 9440–9453.
18. Goli, V. R. (2023). Enabling Intelligent Mobile Experiences with React Native and Machine Learning. International Journal of Multidisciplinary Research in Science, Engineering and Technology, 6(12), 3835-3839. https://doi.org/10.15680/IJMRSET.2023.0612044
19. Zhao, Y., Pan, Y., & Wang, M. (2018). Fault-tolerant storage strategies in distributed cloud systems. Future Generation Computer Systems, 79, 500–509.

View PDF