Seamless storage in molab

With the launch of our GPU access on molab, our community has been making more ambitious notebooks. That has pushed us to improve our infrastructure in response to how usage patterns on molab have changed. We’re rolling out a series of features to help you manage artifacts in molab for better shareability and reproducibility. We are also changing our storage policy to only save files uploaded or created through the marimo file browser for notebooks created after August 26, 2026. Old notebooks retain their data.

TLDR;

• You can now cache expensive downloads with mo.persistent_cache.
• Only data uploaded via marimo’s file browser is retained on shutdown.
• New Google Drive and Hugging Face APIs so you can reliably work with datasets much bigger than the workspace limits.

Data cached with mo.persistent_cache is subject to limitations (such as on size and TTL) that may change at our discretion.

What people are building

The notebooks people run on GPUs don’t look much like the ones from before the launch. We’ve seen a mix of agent-driven autoresearch, fine tuning, and much heavier visualisations on the rise. These workloads are very different to the upload-a-small-csv-and-analyse-it tasks that previously dominated our traffic.

Before, we were treating everything in the workspace as one kind of thing. Your code and your dataset and your trained model shared a limit with the Hugging Face cache, PyTorch checkpoints, the unpacked archive and the virtualenv. As a result we were managing the files as one big monolith without much thought of how they fit into users’ workflow. Our most advanced users reported consistent data loss issues when they (often accidentally) stored too much data in their notebooks, for example after downloading models or large datasets.

So we took a step back, categorized and enumerated all the ways our community is using notebooks, and thought about how best to build features and experiences that can enable them. Marimo’s philosophy has always been to manage complexity for our users so they can focus on what they are doing, rather than how they are doing it. Below are some of the features we’ve built to better enable our community.

Caching from within your notebook

Marimo has a comprehensive caching mechanism (the mo.persistent_cache decorator) that caches the results of computation keyed by function body and arguments. You can use that to cache expensive functions using a compact representation. This both saves you having to write a bunch of code to conditionally download and manage artifacts, as we’ve seen users do, and is smarter in how it uses memory so subsequent notebook loads are much faster.

We saw a lot of notebooks that would download and consume artifacts as shown here. With marimo caching this same code has less boilerplate and in general more efficient

The following cell, from the example notebook above, skips all the work in the function: importing the library, downloading the weights, and running the forward passes.

@mo.persistent_cache  
def embed_corpus(model_id: str, revision: str, texts: tuple[str, ...]):  
    from sentence_transformers import SentenceTransformer
 
    model = SentenceTransformer(model_id, revision=revision)  
    return model.encode(list(texts))  

You cache a few megabytes as opposed to caching hundreds of megabytes and still having to redo the expensive computation. Find out more about how caching works here and refer to our API documentation on how to use it.

Reading data where it lives

We’ve been building better ways to connect remote storage to molab. We’ve noticed that users tend to reuse the same data across different notebooks (and even across different users). Notebooks end up having the same boilerplate, to read the same data, from the same handful of sources. HuggingFace and GoogleDrive account for the vast majority of data sources we’ve seen notebooks read.

Molab ships with fsspec by default, which you can use to connect to remote filesystems. For example, this lets you read a HuggingFace dataset directly into a polars dataframe.

import polars as pl  
from transformers import TrainingArguments, Trainer
 
df = pl.scan_parquet(  
    "hf://datasets/scikit-learn/adult-census-income"  
    "@~parquet/default/train/0000.parquet"  
)
 
training_args = TrainingArguments(  
    output_dir="my-awesome-model",  
    push_to_hub=True,               # Enables automatic uploading  
    hub_model_id="username/custom-repo-name", # Optional custom name  
)
 
trainer = Trainer(  
    model=model,  
    args=training_args,  
    train_dataset=df,  
)
 
# After training finishes, push the final model card and evaluation metrics  
trainer.push_to_hub()  

Crucially, this code defers download and does some query planning to make sure that you’re only ever downloading columns and rows that you need. This is less running time and boilerplate than first downloading the file, then reading it in. Similarly for outputs, if a long run produces something you want to keep, a Hugging Face repo gives you more than the sandbox does: every checkpoint is a commit you can go back to, and a run that dies partway still leaves its finished epochs somewhere reachable.

Google Drive works the same way through gdrive-fsspec, which we build and ship in the image:

from gdrive_fsspec import GoogleDriveFileSystem
 
fs = GoogleDriveFileSystem(auth_kwargs={"use_local_webserver": False})
 
with fs.open("shared-folder/measurements.csv", "rb") as f:  
    df = pl.read_csv(f)  

This creates a connection to your personal drive. If you’d like to use a shared drive, we recommend using a service account and tokens instead.

You can put your tokens in a .env file then read them into the notebook using the python-dotenv package. Use the secrets pane in the developer panel to easily add secrets to your .env file.

Check out our newly released blog post detailing how remote storage works and our API documentation for how to connect to various storage providers.

Persisting only data introduced through the file browser

The most common way we’ve seen users bring their own, ad-hoc data into molab is by using the UI file browser. This is particularly true in education where educators upload small files that students use to complete assignments (described in this case study). Our one, simple rule moving forward is that we’ll only save artifacts uploaded via the file browser. Everything else tends to be indicative of a way we can improve the platform rather than genuine storage growth. Forked notebooks will still carry the data so your notebooks remain shareable and reproducible, which is the part that matters for sharing. Someone who opens your notebook gets the data it runs on.

What this adds up to

All these features address platform shortcomings we’ve seen users overcome by hand-coding workarounds. These features will make molab faster and more reliable for our community and we’ll continue to see how workloads change in response. We’ll also start thinking more deeply about secret management, privacy, and notebook discoverability. Notebooks created before August 26, 2026, will retain all their data. We do encourage users to migrate their data to use persistent cache and remote storage, however.

If you’ve built something on molab that’s running into any of this, we’d like to hear about it. Most of the above came out of reading notebooks people made public.