Observability-Driven SLO Enforcement in High-Throughput Data Infrastructure
Main Article Content
Abstract
High-throughput data infrastructures underpin mission-critical financial, healthcare, and e-commerce applications that require stringent service-level objectives (SLOs) to ensure both performance and reliability. Despite significant advancements in observability platforms, existing SLO enforcement mechanisms remain primarily static and based on pre-determined thresholds and coarse-grained telemetry that fail to account for the high-level dynamism of data workloads. This contribution fills the research gap identified by devising an observability-driven SLO enforcement framework that is specifically tailored for dynamic, high-volume data pipelines. Leverage real-time metrics such as per-stream latency distributions, adaptive throughput metrics, and fine-grained resource consumption traces, our framework continuously optimizes enforcement policies using feedback loops that map observed behavior to user-centric objectives. We present a hierarchical control architecture that integrates lightweight instrumentation agents with data-nodes and a centralized policy engine, thus allowing for both local corrective measures and global adjustments without excessively high overhead. Leverage a combination of simulation and real-world deployment in an open-source streaming platform, we demonstrate that our framework reduces SLO violations by up to 60% compared to static enforcement, all at sub-millisecond decision latency. We also elaborate on implications of our design on scalability, fault tolerance, and multi-tenant fairness, and how observability-derived insights can inform predictive scaling and proactive resource allocation. The results unveil the potential of observability-driven enforcement, setting the stage for self-adaptive data infrastructures that can uphold service commitments under varying load conditions.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
The license allows re-users to share and adapt the work, as long as credit is given to the author and don't use it for commercial purposes.
References
• Chen, L. Y., & Bahsoon, R. (2017). Self-adaptive and sensitivity-aware QoS modeling for the cloud. IEEE Transactions on Software Engineering, 43(5), 453–475. https://doi.org/10.1109/TSE.2016.2608823
• Sudhakar Tiwari. (2023). Biometric Authentication in the Face of Spoofing Threats: Detection and Defense Innovations. Innovative Research Thoughts, 9(5), 402–420. https://doi.org/10.36676/irt.v9.i5.1583
• Cortez, E., Morrey, C., Moschetti, G., & Budiu, M. (2017). Resource central: Understanding enterprise resource management in large-scale cloud platforms. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI '16), 153–168. https://www.usenix.org/conference/osdi16/technical-sessions/presentation/cortez
• Gambi, A., Toffetti, G., Pezze, M., & Dustdar, S. (2020). Kriging-based self-adaptive cloud autoscaling for cost-efficient and SLO-aware resource provisioning. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 14(4), 1–28. https://doi.org/10.1145/3342192
• Gan, Y., Delimitrou, C., & Kozyrakis, C. (2019). The case for machine learning-based autoscaling in cloud platforms. Proceedings of the 2019 Symposium on Cloud Computing (SoCC '19), 103–116. https://doi.org/10.1145/3357223.3362707
• Mao, M., & Humphrey, M. (2016). A performance study on the VM startup time in the cloud. 2012 IEEE 5th International Conference on Cloud Computing (CLOUD), 423–430. https://doi.org/10.1109/CLOUD.2012.103
• Rasmi, A., & Bahsoon, R. (2021). Auto-tuning and self-learning control strategies for cloud elasticity: A reinforcement learning approach. Future Generation Computer Systems, 124, 249–265. https://doi.org/10.1016/j.future.2021.05.007
• Tang, C., Lin, Z., & Wang, Q. (2020). Observability in distributed systems: A survey. ACM Computing Surveys (CSUR), 53(6), 1–35. https://doi.org/10.1145/3417983
• Tirmazi, A., Ousterhout, K., Shen, A., Ghodsi, A., & Zaharia, M. (2020). Borg: The next generation. ACM Queue, 18(3), 30–39. https://queue.acm.org/detail.cfm?id=3399814
• Yadwadkar, N., & Katz, R. (2018). Learning autoscaling policies for cloud applications. Proceedings of the 2018 ACM Symposium on Cloud Computing (SoCC '18), 359–373. https://doi.org/10.1145/3267809.3275399
• Zhang, H., Liu, X., Pu, Q., & Lan, Z. (2019). DADS: A real-time deep anomaly detection system for streaming data. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 1875–1883. https://doi.org/10.1145/3292500.3330690
• Google Cluster Data v2. (2019). Retrieved from https://github.com/google/cluster-data
• Alibaba Cluster Trace Program. (2018). Retrieved from https://github.com/alibaba/clusterdata
• OpenTelemetry Project. (2021). OpenTelemetry documentation. Retrieved from https://opentelemetry.io/docs/
• Prometheus Authors. (2022). Prometheus monitoring system documentation. Retrieved from https://prometheus.io/docs/introduction/overview/
• RedHat Developers. (2021). Introduction to eBPF and observability. Retrieved from https://www.redhat.com/en/topics/linux/what-is-ebpf
• Jaeger Tracing. (2021). Jaeger documentation for distributed tracing. Retrieved from https://www.jaegertracing.io/docs/
• Sloth SLO Framework. (2023). GitHub repository for SLOs as code. Retrieved from https://github.com/slok/sloth
• Reinforcement Learning with RLlib. (2020). RLlib documentation from Ray.io. Retrieved from https://docs.ray.io/en/latest/rllib/
• Munn, R., & Goh, J. (2024). Toward automated SLO-based orchestration in cloud-native systems. Journal of Cloud Computing, 13(1), 55–72. https://doi.org/10.1186/s13677-024-00356-4
• Agrawal, S., & Tripathi, A. (2023). Distributed observability and adaptive enforcement policies in service mesh architectures. IEEE Access, 11, 45523–45539. https://doi.org/10.1109/ACCESS.2023.3265509