Rob Harrison
Ross Teixeira
Jennifer Rexford
ABSTRACT
Detecting "heavy hitter" flows is the core of many network security applications. While past work shows how to measure heavy hitters on a single switch, network operators often need to identify network-wide heavy hitters on a small timescale to react quickly to distributed attacks. Detecting network-wide heavy hitters efficiently requires striking a careful balance between the memory and processing resources required on each switch and the network-wide coordination protocol. We present Carpe, a distributed system for detecting network-wide heavy hitters with high accuracy under communication and state constraints. Our solution combines probabilistic counting techniques on the switches with probabilistic reporting to a central coordinator. Based on these reports, the coordinator adapts the reporting threshold and probability at each switch to the spatial locality of the flows. Simulations using traffic traces show that our prototype can detect network-wide heavy hitters with 97% accuracy, while reducing the communication overhead by 17% and switch state by 38%, compared to existing approaches.
- Yehuda Afek, Anat Bremler-Barr, Shir Landau Feibish, and Liron Schiff. 2018. Detecting heavy flows in the SDN match and action model. Computer Networks 136 (2018), 1--12.Google Scholar
Cross Ref
- Ran Ben-Basat, Xiaoqi Chen, Gil Einziger, Shir Landau Feibish, Danny Raz, and Minlan Yu. 2020. Routing Oblivious Measurement Analytics. In IFIP Networking Conference.Google Scholar
- Ran Ben-Basat, Xiaoqi Chen, Gil Einziger, and Ori Rottenstreich. 2018. Efficient Measurement on Programmable Switches Using Probabilistic Recirculation. In IEEE International Conference on Network Protocols ICNP. 313--323.Google Scholar
- Ran Ben-Basat, Gil Einziger, Shir Landau Feibish, Jalil Moraney, and Danny Raz. 2018. Network-wide routing-oblivious heavy hitters. In Symposium on Architectures for Networking and Communications Systems ANCS. 66--73.Google Scholar
- Theophilus Benson and Balakrishnan Chandrasekaran. 2017. Sounding the Bell for Improving Internet (of Things) Security. In Workshop on Internet of Things Security and Privacy, IoT S&P@CCS. 77--82.Google Scholar
- Yanpei Chen, Rean Griffiths, David Zats, Anthony D. Joseph, and Randy H. Katz. 2012. Understanding TCP Incast and its Implications for Big Data Workloads. ;login 37, 3 (June 2012).Google Scholar
- Benoit Claise. 2004. Cisco Systems NetFlow Services Export Version 9. RFC 3954 (2004).Google ScholarDigital Library
- Graham Cormode. 2011. Continuous Distributed Monitoring: A Short Survey. In International Workshop on Algorithms and Models for Distributed Event Processing.Google ScholarDigital Library
- Graham Cormode, S Muthukrishnan, and Ke Yi. 2011. Algorithms for Distributed Functional Monitoring. ACM Transactions on Algorithms 7, 2 (2011), 21:1--21:20.Google ScholarDigital Library
- Damu Ding, Marco Savi, Gianni Antichi, and Domenico Siracusa. 2020. An Incrementally-Deployable P4-Enabled Architecture for Network-Wide Heavy-Hitter Detection. IEEE Transactions on Network and Service Management 17, 1 (2020), 75--88.Google ScholarDigital Library
- Cristian Estan and George Varghese. 2003. New Directions in Traffic Measurement and Accounting: Focusing on the Elephants, Ignoring the Mice. ACM Transactions on Computer Systems 21, 3 (2003), 270--313.Google ScholarDigital Library
- Rob Harrison, Qizhe Cai, Arpit Gupta, and Jennifer Rexford. 2018. Network-Wide Heavy Hitter Detection with Commodity Switches. In ACM SIGCOMM Symposium on SDN Research SOSR. 8:1--8:7.Google Scholar
- Qun Huang, Xin Jin, Patrick P. C. Lee, Runhui Li, Lu Tang, Yi-Chao Chen, and Gong Zhang. 2017. SketchVisor: Robust Network Measurement for Software Packet Processing. In ACM SIGCOMM. 113--126.Google Scholar
- Sushant Jain, Alok Kumar, Subhasree Mandal, Joon Ong, Leon Poutievski, Arjun Singh, Subbaiah Venkata, Jim Wanderer, Junlan Zhou, Min Zhu, Jon Zolla, Urs Hölzle, Stephen Stuart, and Amin Vahdat. 2013. B4: Experience with a Globally-deployed Software Defined WAN. In ACM SIGCOMM. 74--87.Google Scholar
- Yuliang Li, Rui Miao, Changhoon Kim, and Minlan Yu. 2016. FlowRadar: A Better NetFlow for Data Centers. In USENIX NSDI. 311--324.Google Scholar
- Zaoxing Liu, Antonis Manousis, Gregory Vorsanger, Vyas Sekar, and Vladimir Braverman. 2016. One Sketch to Rule Them All: Rethinking Network Flow Monitoring with UnivMon. In ACM SIGCOMM. 101--114.Google Scholar
- P. Phaal, S. Panchen, and N. McKee. 2001. InMon Corporation's sFlow: A Method for Monitoring Traffic in Switched and Routed Networks. RFC 3176 ( 2001).Google Scholar
- report [n. d.]. The CAIDA Anonymized Internet Traces 2016 Dataset. https://www.caida.org/data/passive/passive_2016_dataset.xml. ([n. d.]).Google Scholar
- Vyas Sekar, Michael K. Reiter, Walter Willinger, Hui Zhang, Ramana Rao Kompella, and David G. Andersen. 2008. cSamp: A System for Network-Wide Flow Monitoring. In USENIX NSDI. 233--246.Google ScholarDigital Library
- Vibhaalakshmi Sivaraman, Srinivas Narayana, Ori Rottenstreich, S. Muthukrishnan, and Jennifer Rexford. 2017. Heavy-Hitter Detection Entirely in the Data Plane. In ACM SIGCOMM Symposium on SDN Research SOSR. 164--176.Google Scholar
- slowloris 2009. Slowloris HTTP DoS. https://web.archive.org/web/20150426090206/http://ha.ckers.org/slowloris. (June 2009).Google Scholar
- url [n. d.]. Barefoot's Tofino. https://www.barefootnetworks.com/technology/. ([n. d.]).Google Scholar
- AnWang, Wentao Chang, Songqing Chen, and Aziz Mohaisen. 2018. Delving Into Internet DDoS Attacks by Botnets: Characterization and Analysis. IEEE/ACM Transactions on Networking 26, 6 (2018), 2843--2855.Google ScholarDigital Library
- Tong Yang, Jie Jiang, Peng Liu, Qun Huang, Junzhi Gong, Yang Zhou, Rui Miao, Xiaoming Li, and Steve Uhlig. 2018. Elastic Sketch: Adaptive and Fast Network-Wide Measurements. In ACM SIGCOMM. 561--575.Google ScholarDigital Library
- Ke Yi and Qin Zhang. 2009. Optimal Tracking of Distributed Heavy Hitters and Quantiles. In ACM SIGMOD-SIGART-SIGACT Symposium on Principles of Database Systems PODS. 167--174.Google Scholar
- Minlan Yu, Lavanya Jose, and Rui Miao. 2013. Software Defined Traffic Measurement with OpenSketch. In USENIX NSDI. 29--42.Google Scholar
Index Terms
- Carpe Elephants: Seize the Global Heavy Hitters
Recommendations
Concerto: cooperative network-wide telemetry with controllable error rate
APSys '20: Proceedings of the 11th ACM SIGOPS Asia-Pacific Workshop on SystemsNetwork-wide telemetry requires real-time analysis of a large amount of traffic. Telemetry systems use stream processors to support various applications, and Protocol Independent Switching Architecture switches to reduce the workload on stream ...
Computing discounted multidimensional hierarchical aggregates using modified Misra Gries algorithm
Finding the "Top k" list or heavy hitters is an important function in many computing applications, including database joins, data warehousing (e.g., OLAP), web caching and hits, network usage monitoring, and detecting DDoS attacks. While most ...
Beating CountSketch for heavy hitters in insertion streams
STOC '16: Proceedings of the forty-eighth annual ACM symposium on Theory of ComputingGiven a stream p1, …, pm of items from a universe U, which, without loss of generality we identify with the set of integers {1, 2, …, n}, we consider the problem of returning all ℓ2-heavy hitters, i.e., those items j for which fj ≥ є √F2, where fj is ...
Comments