Short Tail: taming tail latency for erasure-code-based in-memory systems

Abstract

In-memory systems with erasure coding (EC) enabled are widely used to achieve high performance and data availability. However, as the scale of clusters grows, the server-level fail-slow problem is becoming increasingly frequent, which can create long tail latency. The influence of long tail latency is further amplified in EC-based systems due to the synchronous nature of multiple EC sub-operations. In this paper, we propose an EC-enabled in-memory storage system called Short Tail, which can achieve consistent performance and low latency for both reads and writes. First, Short Tail uses a lightweight request monitor to track the performance of each memory node and identify any fail-slow node. Second, Short Tail selectively performs degraded reads and redirected writes to avoid accessing fail-slow nodes. Finally, Short Tail posts an adaptive write strategy to reduce write amplification of small writes. We implement Short Tail on top of Memcached and compare it with two baseline systems. The experimental results show that Short Tail can reduce the P99 tail latency by up to 63.77%; it also brings significant improvements in the median latency and average latency.

摘要

为获得高性能和高数据可用性,基于纠删码的内存存储系统得到广泛应用。然而,随着集群规模不断增长,服务器级别的性能降级问题出现得越来越频繁,进而导致长尾延迟。在基于纠删码的系统中,由于一个纠删码操作可能依赖于多个子操作的同步完成,长尾延迟的影响被进一步放大。本文提出一种称为ShortTail的基于纠删码的内存存储系统,该系统可实现稳定的性能和较低的读写延迟。首先,ShortTail使用轻量请求监视器监测每个内存节点性能,以便及时发现性能降级节点。其次,ShortTail选择性执行降级读操作和重定向写操作,以避免访问性能降级节点。最后,ShortTail采用一种自适应写策略降低小写请求的写放大程度。本文在Memcached上实现了ShortTail,并将其与两个系统进行比较。实验结果表明,ShortTail最高可降低63.77%的99分位延迟,且显著改善中位延迟和平均延迟。