Monday 28 December 2015

NEC HYDRAstore - the global de-duplicated storage - research

The father of HYDRAstore is a Polish computer scientist Cezary Dubnicki the CEO of 9livesdata.com.

HYDRAstore is the fastest and most scalable backup system on the world. 

To learn more about the system I highly recommend to watch the TechFieldDay:

http://techfieldday.com/appearance/nec-storage-presents-at-storage-field-day-6/ 

You can read the solid summary of this event on Chin-Fah Heoh blog :

http://storagegaga.com/hail-hydra/

However, I don't agree with this conclusion:
 
"deduplication solutions such as HydraStor, EMC Data Domain, and HP StoreOnce, are being superceded by Copy Data Management technology, touted by Actifio."

I believe that HYDRAstore approach is very uniq and Actifio 'Copy Data Managment' seems to be similar - I won't be surprised if Actifio is using the cryptographic hash table as well for their VDP (VirtualData Pipeline) with some 'magic' souse. BTW I am huge supporter of Actifio too but I couldn't find any deep dive materials.

What I really like about the HYDRAstore and 9livesdata is that they realy share real knowledge without any marketing yadda..yadda.. (I know NEC HYDRAstore marketing is quite ancient like their GUI:)

But those White Papers defends itself:


"Reducing fragmentation impact with forward knowledge in backup systems with deduplication" – SYSTOR'15, Haifa, Izrael
"Fuzzy adaptive control for heterogeneous tasks in high-performance storage systems" – SYSTOR'13, Haifa, Izrael
"Concurrent Deletion in a Distributed Content - Addressable Storage System with Global Deduplication" – FAST'13, San Jose, USA
"Reducing Impact of Data Fragmentation Caused By In-Line Deduplication" – SYSTOR'12, Haifa, Izrael
"Anchor-driven subchunk deduplication" – SYSTOR'11, Haifa, Izrael"Bimodal Content Defined Chunking for Backup Streams" – FAST'10, San Jose, USA
"HydraFS: A High-Throughput File System for the HYDRAstor Content- Addressable Storage System" – FAST'10, San Jose, USA 

"HYDRAstor: a Scalable Secondary Storage" – FAST'09, San Francisco, USA
"FPN: A Distributed Hash Table for Commercial Applications" – HPDC'04, Honolulu, USA


the end.