Reading Time: 2 minutes

TECHNOLOGY DEEP DIVE

Global Deduplication & Compression

Panzura CloudFS™ automatically deduplicates and compresses all data before it is synced to the cloud. The result is a dramatically reduced storage footprint from the outset.

Panzura-tech-page-header-abstract-shapes 1

Panzura CloudFS deduplicates data before it’s consolidated in the cloud, across every site in your network. Each unique block from a file is stored once and metadata pointers are updated to record which blocks comprise the file. Here’s how it works:

In-line deduplication

Data is deduplicated in-line, at the most efficient, granular block level when files are created or changed. If a block already exists, metadata references are updated so the data doesn’t need to be replicated.

Globally accessible

Deduplication data is stored with the metadata, so each Panzura node has a full record of the deduplication tables. Even if a file block isn’t local, the deduplication engine uses it to make the entire system more efficient.

Instant Updates

Deduplication data is updated instantly through metadata from other nodes and local write activity. As a result, every location in your network always has an up-to-the-moment view of every file.

Scales to Petabytes

Panzura’s patented technology scales to petabytes and minimizes lookups for unique data, making it efficient and highly performant, even for large volumes of enterprise data.

Compression

After deduplication, Panzura CloudFS uses a lossless compression algorithm to break each file into blocks as it is created. Each block is compressed in-line, in memory, as it’s created. Last but not least, Panzura CloudFS takes advantage of redundancy within files to aid in greater reduction in file size.

Caching Hot Data Locally

In addition to caching metadata and global metadata locally in flash, Panzura uses several additional techniques to ensure a fast and seamless file access experience.

Cache Hot Data

Caches hot data on each node based on read and write frequency.

Policy-based Caching

Provides policy-based caching based on file types, folders, and other criteria.

Optimized Metadata

Optimize metadata to get maximum utilization of each node.

Built-In WAN Optimization

The global deduplication and compression provided in Panzura CloudFS makes it possible to reduce or eliminate expensive WAN optimization appliances and eliminate costly MPLS or other private networks.

This is because

Only unique file data is sent to the cloud when it's created

Only active data is cached locallly

Global file locking keeps file operations local and application data doesn't need to cross the WAN each time a user opens, saves, or closes a file

Latency is no longer a problem since file operations are local

Why Global Deduplication and Compression Matter

Leading Video Game Developer

Problem

1.5 PB of build files across 30 offices
Spending millions of dollars on enterprise NAS systems, mirroring, and backups

Results

Consolidated file data down to 45 TB of storage (99% reduction)
Cloud economics enabled them to pay $4000 a month for all tiers of storage

Download Whitepaper

Accelerate digital transformation with a powerful hybrid cloud file platform that:

Modernizes Storage Architecture

Allows organizations to significantly reduce storage costs by consolidating dispersed file data into a unified, deduplicated, compressed, and secured data set in the cloud or on premises.

Unlocks Organizational Productivity

Enables file data to look and feel local to users and processes everywhere. It uniquely empowers users to harness their collective skills by working collaboratively, regardless of location.

Delivers Seamless Cloud File Services

Turns public or private cloud storage into a high performance, immutable global file system that flawlessly delivers file data to people, processes, and AI, and makes it resilient to damage.

Reduces Operational Complexity

Lets IT teams turn their attention to innovation with less file data and infrastructure to manage and protect, fewer storage refreshes to plan for and less worry about recovering from file damage.

Discover how to dramatically improve file data resilience and productivity while simplifying operations with our powerful hybrid cloud file and data services platforms.