cunoFS joins the distributed cloud

ghanshyam

POSIX isn't going anywhere, so why don't we make POSIX implementations extensible by virtualizing syscalls Here's our proposal and what would need to be done.

Virtualizing Syscalls with Dynasaur: Extending Linux into the Future

Our article Is POSIX Outdated in the Cloud Era? spurred a bit of discussion in the developer community (especially in our native field of high-performance cloud computing), and these discussions have solidified our position: while POSIX within Linux is here to stay, we’re not the only ones who want to innovate beyond what the Linux …

Virtualizing Syscalls with Dynasaur: Extending Linux into the Future Read More »

cunoFS is a Cheaper and Faster Alternative to FSx for Lustre

cunoFS is a Faster and Cheaper Alternative to FSx for Lustre

FSx for Lustre is a distributed parallel filesystem that’s used as part of HPC (high-performance computing) workloads across a variety of complex fields (including life sciences, machine learning, meteorology, finance, and media). It is AWS’s managed version of the Lustre filesystem, and while it’s a popular high-performance storage option, it can quickly become unaffordable as …

cunoFS is a Faster and Cheaper Alternative to FSx for Lustre Read More »

How to migrate your on-premises scientific workloads to Azure

How to migrate your on-premises scientific workloads to Azure

Migrating on-premises scientific workloads to the cloud has many benefits in terms of cost and performance that are attractive to both academic and business organizations. Microsoft Azure is a popular cloud platform within the scientific research community as it offers a variety of features and tools to build and scale the infrastructure for your workflows, …

How to migrate your on-premises scientific workloads to Azure Read More »

High-Performance Object Storage — Fast, Infinitely Scalable Storage for HPC and Scientific Workloads

High-Performance Object Storage — Fast, Infinitely Scalable Storage for HPC and Scientific Workloads

High-performance computing, machine learning (ML) and AI, and scientific workloads must be built to be scalable, a requirement that is often difficult for traditional filesystem storage solutions to meet. Object storage provides a fast, networked, and highly scalable storage option for these use cases, but there are some inherent performance and compatibility problems that need …

High-Performance Object Storage — Fast, Infinitely Scalable Storage for HPC and Scientific Workloads Read More »

Mountpoint-S3 Alternatives: POSIX-Compatible, High-Performance S3 for Linux and Serverless in AWS

Mountpoint-S3 Alternatives: POSIX-Compatible, High-Performance S3 for Linux and Serverless in AWS

Mountpoint for Amazon S3 connects AWS S3 buckets to Linux filesystems. While intended to bridge the gap between object storage and filesystems (and Kubernetes using a CSI driver), it has several downsides that cause problems for developers: it has limited POSIX compatibility, limited directory handling and file editing, and inconsistent file writing behavior. This article …

Mountpoint-S3 Alternatives: POSIX-Compatible, High-Performance S3 for Linux and Serverless in AWS Read More »

Affordable and Fast: S3 is Now the Best Place to Process your FFmpeg Data

Affordable and Fast: S3 is Now the Best Place to Process your FFmpeg Data

Most companies that use FFmpeg and store their data in object storage (such as S3, Azure, Google Cloud, or on-premises solutions like MinIO) stage their data to a filesystem, then run FFmpeg on it there, and finally upload the result back into object storage. This is because FFmpeg requires POSIX file access, and existing POSIX …

Affordable and Fast: S3 is Now the Best Place to Process your FFmpeg Data Read More »

Making the Right Choice: Comparing the Cost/Performance of Different EFS Options and Alternatives

When choosing the EFS (Elastic File System) configuration options for your enterprise cloud storage, the choices can be overwhelming. Practical information can be hard to find, so we’ve assembled all of the relevant facts in one place, along with additional information that will help you pick the right mix of price and performance for your …

Making the Right Choice: Comparing the Cost/Performance of Different EFS Options and Alternatives Read More »

Optimizing PyTorch Machine Learning Cost and Performance Using cunoFS

Optimizing PyTorch Machine Learning Cost and Performance Using cunoFS

Performance and cost are important considerations when designing and implementing machine learning (ML) systems at scale. The speed and efficiency with which you can train your ML models and extract the maximum value from your data will determine the overall success of your project. Data loading is one of the biggest performance factors in the …

Optimizing PyTorch Machine Learning Cost and Performance Using cunoFS Read More »