File data typically contains a LOT of duplication. Users make endless copies so you end up storing similar or even identical files multiple times. Backup and disaster recovery processes make even more copies. That creates storage bloat, it's a win for your storage vendor, and a painfully expensive (and continually growing) storage bill for you.
The CloudFS hybrid cloud file platform takes a radically different approach. It's laser-focused on reducing the amount of data stored by squeezing out every last redundant byte of data, everywhere. Even at enterprise scale, that can be up to 80% of the total volume of data you're currently paying to store and back up.
Global Scope: Other deduplication solutions operate only within a single site or storage volume. That's better than nothing but it's far from as efficient as it could be. With CloudFS, deduplication is global. This means it identifies and eliminates duplicate data across all locations and in the central object storage.
Inline Deduplication: We like to say that only immediate is fast enough and that applies to everything we do with data. CloudFS performs deduplication inline — as data is being written and changed, it's compared to data that already exists. This is more efficient than post-process deduplication, which processes data after it's been written, as it prevents redundant data from ever being stored in the first place.
Block-Level Deduplication: CloudFS doesn't just look for duplicate files. It translates files into data blocks that are a tiny 128kb in size (that's as small as it gets) and creates metadata pointers that record which blocks make up the file in that moment. It then compares these individual blocks. If an identical block already exists anywhere within the global file system, it creates and stores a metadata pointer to that block. Instead of storing redundant megabytes, you're adding lightweight metadata.
Shared Metadata and Deduplication Reference Table: We do really smart things with metadata and this is the piece that means you squeeze out every last byte of data across your entire file system. Our deduplication reference table, which tracks unique data blocks and their locations, is embedded in the metadata that is instantly shared among all Panzura nodes. This means that every location has an up-to-date view of all unique data blocks stored globally.
Object Store as the Single Source of Truth: Panzura consolidates all data into a single, authoritative dataset in your chosen cloud object storage — AWS S3, Azure Blob Storage, Google Cloud Storage and others or on-premises object storage, for example Nutanix Objects, Cloudian and others. This object storage then acts as the central repository for the deduplicated data.
