Global Deduplication and Compression
As an enterprise file system, Panzura CloudFS™ automatically performs global deduplication and compression on all data before it is sent over the wire. The result is reduced network traffic and a smaller storage footprint, reducing costs and increasing performance.
Other storage systems deduplicate data locally, but can’t manage data deduplication across sites. Panzura CloudFS deduplicates data before it’s consolidated in the cloud. Each unique block from a file is stored once, so only one unique copy of a file is preserved by the file system.
Here is how Panzura deduplication works:
Data is deduplicated in-line when files are created or changed. If a file block already exists, metadata references are created so the data doesn’t need to be written.
Globally accessible deduplication data
Deduplication data is stored with the metadata, so each Panzura Filer has a full record of the deduplication tables. Even if a file block isn’t local, the deduplication engine uses it to make the entire system more efficient.
Deduplication data is updated instantly through metadata from other Filers and local write activity.
Scales to petabytes
Panzura patented technology scales to petabytes and minimizes lookups for unique data, making it efficient even for large volumes of data.
After deduplication, Panzura CloudFS compresses file data as it’s created. Panzura CloudFS uses a lossless compression algorithm and breaks each file into blocks as it is created. Each block is compressed in-line, in memory, as it’s created. Last but not least, Panzura CloudFS takes advantage of redundancy within files to aid in greater reduction in file size.
Caching Hot Data Locally
In addition to caching metadata and global metadata locally in flash, there are several additional techniques used to ensure a fast file access experience.
Cache hot data
Caches hot data on each Freedom Filer based on read and write frequency
Provides policy-based caching based on file types, folders, and other criteria
Optimize metadata to get maximum utilization of each Freedom Filer
Built-In WAN Optimization
The global deduplication and compression provided in Panzura CloudFS makes it possible to reduce or eliminate expensive WAN optimization appliances and eliminate costly MPLS or other private networks. This is because (1) only unique file data is sent to the cloud when it’s created; (2) only active data is cached locally; (3) global file locking keeps file operations local and application data doesn’t need to cross the WAN each time a user opens, saves, or closes a file; and (4) latency is no longer a problem since file operations are local.
Why Global Deduplication and Compression Matter
Leading Video Game Developer
- 1.5 PB of build files across 30 offices
- Spending millions of dollars on enterprise NAS systems, mirroring, and backups
- Consolidated down to 45 TB of storage (99% reduction)
- Cloud economics enabled them to pay $4000 a month for all tiers of storage