Processing gi-normous data sets like we often do at the ANML has required writing quite a bit of custom code, and we're getting ready to make some of that software available to a wider audience. One of the first bits will be a set of libraries for searching very large collections of Netflow data with boolean queries and pretty-printing the output in the format of your choice.
The "flowseek" system doesn't offer quite the flexibility of SQL, but it's nice to be able to quickly select, say, all the flows with a total size between 15,000 and 20,000 bytes, from a particular prefix, destined to either port 9001 or 9002, but not from that one host you already know about. And it's nice and fast, since it actually writes out your query as C code, compiles it, and imports it again as a shared object module!
More details later...
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment