We've been using klauspost's Golang reedsolomon package in our distributed storage package. It runs approximately as fast (~1GB) because the hot paths are written in asm: https://github.com/klauspost/reedsolomon

Erasure codes are pretty magical.

Firstly this looks cool and I'll be looking at it closer when I get home from work.

> Note that all speeds mentioned here are measured on i7-4770, employing all features available in a particular program - including multi-threading, SIMD and x64 support.

I hope this doesn't wind up infringing on the Streamscale patent[1] as some other works have in this field. Again I need to give this a proper read.

[1] https://www.google.com/patents/US8683296

I've implemented a Reed-Solomon encoder for telemetry blocks over a RF channel. There, I was using a (255, 223) code on GF(2^8). There were interleaving parameters for a telemetry "packet" but each encoding took constant O(1) time. So the overall time to encode a downlink was just O(N). What exactly is meant by "Reed-Solomon ECC"? I'm guessing the code is doing something different related to data storage than what a traditional communications RS code does?

Anyone compared this to the intel ISA-L library that uses the various intel specific instructions to accelerate reed solomon encoding?

I believe this is what Facebook, Microsoft, and similar organizations are going to avoid Hadoops 3x replication overhead, while still keeping the same degree of protection.

https://github.com/01org/isa-l

Are there any good explanations of Reed-Solomon? The mathematics look a bit hairy..

For a pretty quick implementation w/ good C/C++ APIs, as well as BCH decoding, check out https://github.com/pjkundert/ezpwd-reed-solomon

Also quickly available as a JavaScript and Python binding...

I'm always confused about all those codes (RS, LDPC, Turbo etc.).

If you have the parchive use case, i.e. you want to "armor" files on disk so that you can detect and correct bit flips (including bursts), which codes are useful and which aren't?

RS obviously is (parchive, optical disks etc.). But are fountain codes usable? Low density parity codes? I have no idea how I would select a code family.