Exploring The Synergy of XAAS and Lambda Architecture for Efficient Data Stream Processing in Cloud Systems

Paper ID: EIJTEM_2015_2_4_61-74

Author's Name: Narasimha Rao Boinapalli

Volume: 2

Issue: 4

Year: 2015

Page No: 61-74

Abstract:

Real-time analytics, along with architectures capable of processing large amounts of data, have become increasingly necessary and crucial due to the phenomenon of big data. In this paper, we will describe the Service (XaaS) and Lambda Architecture for optimizing data stream processing in the cloud. XaaS provides an on-demand, scalable cloud solution, while Lambda Architecture performs large-scale data processing in both batch and real-time modes. Existing and Lambda-based models will continue to struggle with latency, high operating costs, and a lack of elasticity in the face of large-scale data streams. The model, which proposes X-Lambda Stream, aims to address the challenges of integrating XaaS with Lambda Architecture by cloudifying them to achieve better elasticity, resource control, and cost-effectiveness. X-Lambda Stream features an enhanced processing layer that enables improved real-time processing by reducing latency through dynamic load balancing and resource provisioning, along with advanced fault tolerance, ensuring system reliability. The more robust infrastructural costs, inefficiencies in handling high-velocity data, and data consistency issues of traditional Lambda models are also outlined. The model that XaaS and Lambda’s Architecture’s hybridization proposes solves the problems that have been outlined and additionally provides superior methods of real-time stream processing in cloud-based systems. The effectiveness of this model in improving data processing speed, reducing costs, and providing real-time insights across multiple cloud applications is articulated in the studies.

Keywords: Xaas, Lambda Architecture, Data Stream Processing, Cloud computing, Real-time analytics, Scalability, Fault Tolerance, Cloud Native Services, Big data processing

References:

1. De Oliveira, F. A., Ledoux, T., & Sharrock, R. (2013, September). A framework for the coordination of multiple autonomic managers in cloud environments. In 2013 IEEE 7th International Conference on Self-adaptive and Self-organizing Systems (pp. 179-188). IEEE.
2. Pavić, I. (2013). Development of the international standards for hydrographic surveys. In ZBORNIK RADOVA CONFERENCE PROCEEDINGS (p. 100).
3. Hasani, Z., Kon-Popovska, M., & Velinov, G. (2014). Lambda architecture for real time big data analytic. ICT Innovations, 133-143.
4. Villari, M., Celesti, A., Fazio, M., & Puliafito, A. (2014, November). AllJoyn Lambda: An architecture for the management of smart environments in IoT. In the 2014 International Conference on Smart Computing Workshops (pp. 9-14). IEEE.
5. Jejkal, T., Hartmann, V., Stotzka, R., Otte, J., Garcia, A., van Wezel, J., & Streit, A. (2012, February). LAMBDA--The LSDF Execution Framework for Data Intensive Applications. In 2012 20th Euromicro International Conference on Parallel, Distributed and Network-based Processing (pp. 213-220). IEEE.
6. Amiri, M., &Akkasi, A. (2015). Assessing security challenges in online social networks. International Academic Journal of Science and Engineering, 2(1), 1–10.
7. Guderian, F., & Fettweis, G. (2011). The lambda chart: A model of design abstraction and exploration at system-level. In Proceedings of International Conference on Advances in System Simulation (SIMUL) (pp. 7-12).
8. Kumar, R., Gupta, N., Charu, S., & Jangir, S. K. (2014, August). Manage big data through newsql. In National Conference on Innovation in Wireless Communication and Networking Technology–2014, Association with THE INSTITUTION OF ENGINEERS (INDIA). sn.
9. Meyer, J. R., Dobias, D. T., Weitz, J. S., Barrick, J. E., Quick, R. T., & Lenski, R. E. (2012). Repeatability and contingency in the evolution of a key innovation in phage lambda. Science, 335(6067), 428-432.
10. Kondori, M. A., & Peashdad, O. H. (2015). Analysis of challenges and solutions in cloud computing security. International Academic Journal of Innovative Research, 2(1), 20–30.
11. Heinze, T., Aniello, L., Querzoni, L., & Jerzak, Z. (2014, May). Cloud-based data stream processing. In Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems (pp. 238-245).
12. Ghattamaneni, Dileep Kumar, and Narasimha Rao Boinapalli. "Performance Optimization of Machine Learning Pipelines on Apache Spark for Real-Time Big Data Analytics." International Journal of System Design and Information Processing, vol. 1, no. 3, 2013, pp. 1-6.
13. Gulisano, V., Jimenez-Peris, R., Patino-Martinez, M., & Valduriez, P. (2010, June). Streamcloud: A large scale data streaming system. In 2010 IEEE 30th International Conference on Distributed Computing Systems (pp. 126-137). IEEE.
14. Kleiminger, W. (2010). Stream processing in the cloud. Imperial College of Science, Technology and Medicine Department of Computing, 1-91.
15. Ji, C., Li, Y., Qiu, W., Awada, U., & Li, K. (2012, December). Big data processing in cloud computing environments. In 2012 12th international symposium on pervasive systems, algorithms and networks (pp. 17-23). IEEE.
16. Selvan, C., Senthil Kumar, R., Iwin Thanakumar Joseph, S., Malin Bruntha, P., Amanullah, M., & Arulkumar, V. (2025). Traffic Prediction Using GPS Based Cloud Data Through RNN-LSTM-CNN Models: Addressing Road Congestion, Safety, and Sustainability in Smart Cities. SN Computer Science, 6(2), 159.
17. Heinze, T., Roediger, L., Meister, A., Ji, Y., Jerzak, Z., & Fetzer, C. (2015, August). Online parameter optimization for elastic data stream processing. In Proceedings of the sixth ACM symposium on cloud computing (pp. 276-287).
18. Cugola, G., & Margara, A. (2012). Processing flows of information: From data stream to complex event processing. ACM Computing Surveys (CSUR), 44(3), 1-62.
19. Zaharia, M., Das, T., Li, H., Shenker, S., & Stoica, I. (2012). Discretized streams: an efficient and {Fault-Tolerant} model for stream processing on large clusters. In 4th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 12).
20. Codescu, M. (2010, July). Lambda expressions in CASL architectural specifications. In International Workshop on Algebraic Development Techniques (pp. 98-117). Berlin, Heidelberg: Springer Berlin Heidelberg.
21. Miyamoto, T., Tanaka, J., Otsuki, H., Kudoh, T., Takano, R., Hayashi, M., ... & Shimojo, S. (2010, December). Field demonstration of dynamic circuit provisioning by web services interface proxy for lambda and ethernet based administrative domains. In Asia Communications and Photonics Conference and Exhibition (p. 79890F). Optica Publishing Group.
22. Chien, A. A., & Taesombut, N. (2009). Integrated resource management for lambda-grids: The Distributed Virtual Computer (DVC). Future Generation Computer Systems, 25(2), 147-152.
23. Tsukishima, Y., Sameshima, Y., Hirano, A., Jinno, M., Kudoh, T., & Okazaki, F. (2008, September). Multi-layer lambda Grid properly using lambdas and sub-lambdas. In 2008 34th European Conference on Optical Communication (pp. 1-2). IEEE.
24. Cao, Y., Lu, H. M., & Liang, J. (2010). Probability landscape of heritable and robust epigenetic state of lysogeny in phage lambda. Proceedings of the National Academy of Sciences, 107(43), 18445-18450.
25. Griwodz, C., Liepert, M., Zink, M., & Steinmetz, R. (2000). Tune to lambda patching. ACM SIGMETRICS Performance Evaluation Review, 27(4), 20-26.

View PDF