Implementing cost-effective, scalable storage has traditionally presented problems to developers. While object storage is cheap and scalable, it requires that existing tools be adapted to work with cloud providers’ APIs rather than the filesystem.
Historically, tools that bridge the gaps between object storage and POSIX environments have suffered limitations — particularly poor performance, scalability, or the need to store data in proprietary formats — that make them unsuitable for high-performance use cases.
With cunoFS, you can interact with object storage as if it were a native filesystem, with high performance (up to 50x faster than native access!) and POSIX compatibility that can run almost any new or existing application.
What is cunoFS?
cunoFS is a scalable, high-performance POSIX compatibility layer that lets you interact with files stored on object storage such as Amazon S3, Azure Blob Storage, Google Cloud Storage, or any other S3-compatible object store hosted in the cloud or locally. This lets you:
- Reliably read, write, and execute files in object storage as if they were stored on your native filesystem.
Move big data and high-performance computing workloads to more affordable object storage.
Stop re-writing POSIX applications to use object storage APIs and temporarily staging data on intermediate storage for use.
Other tools in this space require developers to make tradeoffs between native filesystem access, POSIX compliance, and high performance.
cunoFS originated as the solution for genomics researchers at PetaGene. PetaGene’s customers, which include some of the world’s most prominent healthcare and research institutions, needed to access large volumes of data on object storage efficiently, while maintaining compatibility with their existing applications. cunoFS achieved this with incredible success, and PetaGene is supporting the further development and release of this technology to all developers who will benefit from it.
How cunoFS works
cunoFS works by providing a transparent compatibility layer that presents object storage as a native filesystem. Filesystem access is intercepted and translated to object storage API calls.
Our compatibility layer is heavily optimized for fast directory listing, predicts application behavior to accelerate performance, and even allows for the retention of POSIX metadata on object storage. All of this is achieved with no lock-in — all files stored on S3-compatible storage can be read back without cunoFS, directly from the buckets they were written to.
Adaptable between cloud, local, and hybrid environments
Every organization has its own mix of cloud and on-premises infrastructure and performance requirements. One of the primary goals of cunoFS is to allow you to keep doing things the way that works best for you, so we include several modes of operation so that cunoFS can function optimally regardless of how you access data between cloud and local deployments and the level of POSIX compatibility you require.
By default, cunoFS is configured and optimized to be used in a LAN setting, on which all infrastructure is on the same high-performance network — for example, in the same datacenter or local network. This configuration maintains POSIX metadata for files: permissions, symlinks and hardlinks continue to function at the POSIX level, with this data being stored in hidden files at the object-storage level.
Configuration options are also available for optimizing for access over WAN networks, where the data is either stored or accessed from a remote network. POSIX metadata is discarded to improve performance, and local caching options are available when unpredictable write positions are likely — for example, when editing videos.
By default cunoFS intercepts calls to standard file access library functions, injecting code that provides access to object storage with URI-based access. This is highly performant and can be done entirely without elevated system privileges. Alternatively, cunoFS can be used via a FUSE mount for path-based access for use cases that don’t support intercepts. A combined mode is also available that allows both.
Seamlessly combine fast EFS storage with S3 object storage
Fusion cunoFS provides a “best of both worlds” approach to performant elastic and cost-effective object storage. It exposes a hybrid of any shared filesystem (including cloud filesystems like AWS EFS, Azure Files, and Google Cloud Filestore) and S3-compatible object storage for use in POSIX environments. It provides a high-IOPS read/write hybrid file store with data being automatically shifted between the two based on access frequency and predicted usage.
We’ll be exploring use case scenarios and best practices, and providing benchmarks for each of these modes of operation through upcoming posts announced on our LinkedIn and through our mailing list. Be sure to subscribe!
How does cunoFS benefit you?
Developers benefit from using cunoFS as it reduces the work they need to do to make their applications compatible with object storage. Businesses benefit from this with reduced storage costs and development teams that have more time to solve problems rather than re-implementing existing functionality.
During the development of cunoFS, we focused on features that would ensure tangible, real-world benefits for the following groups and use cases:
- ML developers and researchers who need scalable, affordable cloud storage for their data and models that is also readily accessible.
- IT teams who rely on cloud filesystems such as EFS but are looking for cost and performance improvements.
Users of other object storage gateways who are tired of bottlenecks.
Organizations who want the widest compatibility for their applications — accessing object storage as a native filesystem reduces lock-in, as you can move to (or from) any other POSIX-compatible solution.
Of course, targeting these groups doesn’t mean these benefits are only for them. Many other industries have big data and high-performance computing workloads that will be made vastly more efficient when backed by fast object storage. These benefits come at no performance cost — instead, you will in fact see large increases in data throughput.
Start using cunoFS today
Free for personal use and evaluation
cunoFS is free for personal use — you can register here to download the package and start using it immediately. We’ll provide you with a quick start guide tailored to your use case, and you’ll be free to use cunoFS with your own object storage for as long as you like, within our personal usage limitations.
We offer a 90-day trial for professional and commercial users, so that you can fully test cunoFS before investing in it. This full trial covers all functionality, including cunoFS Fusion for users wishing to replace EFS with an affordable alternative.
cunoFS professional and commercial pricing
During your trial, we will work with you to assess your usage requirements and agree on a licensing structure that provides the best cost savings over pure EFS for your use case. Pricing is usage-based, and we offer a cluster plan for cases that require launching multiple EC2 instances for temporary jobs.
| Personal + | Professional | Enterprise | |
| 0–250 TB | Free (up to 100 TB) | $12/TB/month | |
| next 750 TB (up to 1 PB) | $3/TB/month | ||
| > 1 PB | Contact us |
| Personal + | Professional | Enterprise | |
| Personal non-commercial use within fair-use limits | |||
| Personal non-commercial use beyond fair-use limits | |||
| Commercial | |||
| Academic projects < $1m budget | |||
| Academic projects > $1m budget |
Professional and commercial users see 67% savings vs. pure EFS
| Storage capacity | EFS (price per month)* | cunoFS hybrid EFS + S3 (price per month)* | Amount saved by using cunoFS instead of EFS | % savings by using cunoFS |
| 5 TB | $400 | $133 | $267 | 67% |
| 20 TB | $1,600 | $532 | $1,068 | 67% |
| 250 TB | $20,000 | $6,610 | $13,390 | 67% |
| 500 TB | $40,000 | $10,960 | $29,040 | 73% |
| 750 TB | $60,000 | $15,260 | $44,740 | 75% |
| 1 PB | $80,000 | $19,560 | $60,440 | 76% |
Object storage affordability with filesystem access
A product launch isn’t the final step in the development process — it’s one of the first. We’re really excited about what we’ve achieved with cunoFS, and we think it will become a big part of the foundations for novel solutions in innovative fields such as machine learning, media, and life sciences.
We want to see what you can do with the combination of practically limitless object storage and high-performance native access. You can test cunoFS right now in almost any Linux environment by following our quick start guide and running your own benchmarks.
