Sketching Big Network Data

讲座内容：The Internet has moved into the era of big network data. It presents both opportunities and technical challenges for traffic measurement functions, which have important applications in intrusion detection, resource management, billing and capacity planning, as well as big data analytics. Due to the practical need of processing network data in high volume and at high speed, past research has strived to reduce the memory and processing overhead when measuring a large number of flows. One important thread of research in this area is based on sketches, such as the FM sketches, the Log Log sketches, and the Hyper Log Log sketches. Each sketch requires multiple bits and many sketches are needed for each flow, which results in significant space overhead. In this talk, we present a new method of virtual sketches to summarize big network data into extraordinarily small size. The new method compresses big data into a space of less than 1 bit per flow. Yet it supports extraction of per-flow statistics from such a small summary with good accuracy. We also show how this virtual-sketch research can be extended along/space/time/function/application dimensions.