Is the reload module very different than the base watch flag, and does the notebook editor use PyCharm/JetBrains' editor interfac, or is the notebook's editor completely embedded in PyCharm?
---
As usual on marimo posts I have to mention how much I enjoy the product:
Two weeks ago I was working on a large scale data reconciliation that was very high in complexity and high risk for correctness/accuracy.
I built a marimo notebook to interactively visualize and validate the final data, which lead to finding several edge cases that unit testing and validation queries alone would easily miss.
The notebook used tabbed controls for switching subviews, custom AnyWidget components for a few advanced filters, interactive Altair charts with reactive data selection for drilling-down, and tables to export examples to Google Sheets. So close to the full gamut of features I think.
The more traditional notebook or script based workflows would not have been sufficient, a well-organized reactive notebook saved a lot of time on iterating on components without losing data, and data without having to re-run the full notebook.
Big thanks to Akshay and team, you're doing awesome work! Genuinely this has improved my workflow.
The current version of the extension embeds the marimo user interface in PyCharm. We may in the near future have a more native-feeling embedding, but that's a little tbd.
Glad to hear you're enjoying marimo, and thanks for sharing the anecdote. I'll gladly accept the "thank you", but it's worth mentioning this extension was developed end-to-end by Kiran :)
Yup! Not affiliated in anyway, just a fan. I like jupyter but marimo's my go to for a few reasons:
- Automatic reactivity can be turned on (will trigger dependent cells if you uptade an upstream one)
- Much nicer non-json file format (just python files, with a seperate output)
- Much better virtual environment integration / managent (this is always a hassle with jupyter)
- Widgets are great!
- Better LLM integration (new marimo-pair means llm can inspect the actual python runtime objects)
- It generally "looks" a lot nicer if you're using the web ui, which is nice.
The cell DAG enforces that there’s no implicit state, which reduces cognitive load for me a lot and provides some pressure to abstract experimental code into functions. In Jupyter this is left to user discipline and restart-and-run-all workflow
The reactive components are also really nice for interactive plotting and exploratory data analysis. You can do this in Jupyter but it feels less seamless somehow. Interactive marimo workflow feels like streamlit but in a notebook interface
One thing I miss from Pluto.jl workflow is `let` for lowering friction for exploratory or plot cells. In marimo you have to name a `_` prefixed function and then call it which is better than nothing but not as clean as `let`. This is a minor complaint that’s more down to language features though
Give it a shot, it's not in beta. It's much more powerful and fun to use than `--watch`. Works best with frontier models but is compatible with open source / local models too. If you have feedback please let me know!
I do that a decent chunk of the time yeah especially for learning. I also have a bunch of marimo notebooks that double as clis and they're lovely.
But sometimes I want to do something too specific or high fidelity and it's just easier to get the clanker to write typescript and make a webpage/components.
As an SRE/ops (and I'm talking some years ago) it was often a pain point, especially with devs with a "works on my machine" attitude ; you would never get the exact requirements from them, I even saw a team of 2 who worked together with different versions of stuff on their respective machines. I also worked in some "offline" environments where once in prod you cannot pull anything from the internet, and you cannot install a compiler too, so it's quite hard to ship because some pip packages require to be built.
I found a way using (can't remember the tool name) which if you loop through the imports and gives it to the tool you get the package name, then I would build wheels to have all binaries and build a container or a VM with all that's needed, thus working completely around python package managers. This was a good enough workflow for the kind of deployment we needed.
Jupyter added a layer of complexity, I deployed it alongside RStudio as browser IDEs in docker swarm. Everybody wants a different set of deps and versions, so you have to keep track of everything, and also people may use things just for development that must not be shipped to prod, so you have to keep track of that too. Also some would develop notebooks on windows and expect them to work in linux VMs/containers and even in prod.
Nowadays devs ship container images anyway through a CI so it is less of an issue. In this era docker was far from being the de-facto everywhere, some people were still afraid of this, security didn't like it, etc.
I was interested in marimo, but I became less interested when I realized that they traded off being able to assign to a variable more than once in order to allow out-of-order execution of cells.
I mean as a Jupyter user, I typically do both and just keep track of what I'm doing in my head (like a repl with many snippets I can run any time), but if I wanted to make it more predictable, I would definitely give up out-of-order execution first.
Yes I understand it's a tradeoff, but I'm saying I would prefer a different tradeoff. It would be more intuitive to me if running a cell always invalidated the cells below; this would make variable reassignment unambiguous, just like in a script, but still with all the visualization goodies of a notebook.
Just wanted to mention that you're always able to do this:
for _x in range(100):
...
This way, `_x` is detected as a throwaway Python variable. And it won't re-appear in other cells.
Also, within the same cell you can always re-assign. But you can't do that in another cell. We want to ensure that a variable is fully declared in one, and only one, cell.
My biggest problem with Jupyter is hidden state. You have no idea what order the cells executed in and how many times to get to the current state. Pluto.jl and Marimo solve that by using reactivity to make state transparent. WYSIWYG.
I have also been thoroughly impressed how Marimo has engaged with AI agents. marimo-pair is fantastic.
We have adopted marimo notebook for doing modern data science. I’m happy with their progress. Nice to see greater integration.
What’s “modern” data science?
The kind of data science that you do with marimo notebooks
You're being facetious, right? How is it different from regular data science?
2-player games, 60 card decks, all sets are legal, no more than four of any card in each deck.
Is the reload module very different than the base watch flag, and does the notebook editor use PyCharm/JetBrains' editor interfac, or is the notebook's editor completely embedded in PyCharm?
---
As usual on marimo posts I have to mention how much I enjoy the product:
Two weeks ago I was working on a large scale data reconciliation that was very high in complexity and high risk for correctness/accuracy.
I built a marimo notebook to interactively visualize and validate the final data, which lead to finding several edge cases that unit testing and validation queries alone would easily miss.
The notebook used tabbed controls for switching subviews, custom AnyWidget components for a few advanced filters, interactive Altair charts with reactive data selection for drilling-down, and tables to export examples to Google Sheets. So close to the full gamut of features I think.
The more traditional notebook or script based workflows would not have been sufficient, a well-organized reactive notebook saved a lot of time on iterating on components without losing data, and data without having to re-run the full notebook.
Big thanks to Akshay and team, you're doing awesome work! Genuinely this has improved my workflow.
The reloading mechanism in the blog exercises the module reloader (https://docs.marimo.io/guides/editor_features/module_autorel...), to reload other files (not the notebook file), which is different from the watch flag.
The current version of the extension embeds the marimo user interface in PyCharm. We may in the near future have a more native-feeling embedding, but that's a little tbd.
Glad to hear you're enjoying marimo, and thanks for sharing the anecdote. I'll gladly accept the "thank you", but it's worth mentioning this extension was developed end-to-end by Kiran :)
Any former Jupyter users using Marimo who have the sales pitch? I'm decently happy using Jupyter Lab but would love to hear what I'm missing out on
Yup! Not affiliated in anyway, just a fan. I like jupyter but marimo's my go to for a few reasons:
- Automatic reactivity can be turned on (will trigger dependent cells if you uptade an upstream one) - Much nicer non-json file format (just python files, with a seperate output) - Much better virtual environment integration / managent (this is always a hassle with jupyter) - Widgets are great! - Better LLM integration (new marimo-pair means llm can inspect the actual python runtime objects) - It generally "looks" a lot nicer if you're using the web ui, which is nice.
The cell DAG enforces that there’s no implicit state, which reduces cognitive load for me a lot and provides some pressure to abstract experimental code into functions. In Jupyter this is left to user discipline and restart-and-run-all workflow
The reactive components are also really nice for interactive plotting and exploratory data analysis. You can do this in Jupyter but it feels less seamless somehow. Interactive marimo workflow feels like streamlit but in a notebook interface
One thing I miss from Pluto.jl workflow is `let` for lowering friction for exploratory or plot cells. In marimo you have to name a `_` prefixed function and then call it which is better than nothing but not as clean as `let`. This is a minor complaint that’s more down to language features though
marimo dev here, the pitch that got me working there is widgets!
This video highlights some fancy tricks: https://www.youtube.com/watch?v=qVSeOr3AIbc
this is a very engaging and well presented video, respect to whoever this guy who made it is (cant tell from the description), he's good at his job
Vincent Warmerdam. Check out his PyData talks and the probably.ai YouTube channel for which he has done videos.
Vincent D. Warmerdam ;) there are a bunch of Dutch people who otherwise have the same name.
People also know me from the calmcode youtube channel and https://calmcode.io website
Wow. This is incredible.
This looks like a step up from Jupyter notebooks!
I fucking love marimo for exploring data.
However my use of it has decreased a little with how easily I can conjure disposable frontends with agents to explore one off things.
You might like https://marimo.io/pair, it turns marimo into less of a notebook and more of a shared data/computational canvas for you and your agent
Last I heard, the feature was in beta so avoided it. But I'll definitely give it a go if it's mature now!
I have been using the --watch flag to let my agent play with the notebook as I use it already.
Give it a shot, it's not in beta. It's much more powerful and fun to use than `--watch`. Works best with frontier models but is compatible with open source / local models too. If you have feedback please let me know!
Any support for windows/Powershell yet?
Yes, but we might have bugs which we are prioritizing asap. If you run into issues please file a bug?
https://github.com/marimo-team/marimo-pair/issues
stay tuned this week :)
huh, I was doing this but without the feature. This is neat!
Why not make the agents make what you want with marimo?
I do that a decent chunk of the time yeah especially for learning. I also have a bunch of marimo notebooks that double as clis and they're lovely.
But sometimes I want to do something too specific or high fidelity and it's just easier to get the clanker to write typescript and make a webpage/components.
Not sure about this environment complaint with Jupyter.
I taught venv and pip for years without issues to thousands.
Now I teach uv and it is even faster and easier.
As an SRE/ops (and I'm talking some years ago) it was often a pain point, especially with devs with a "works on my machine" attitude ; you would never get the exact requirements from them, I even saw a team of 2 who worked together with different versions of stuff on their respective machines. I also worked in some "offline" environments where once in prod you cannot pull anything from the internet, and you cannot install a compiler too, so it's quite hard to ship because some pip packages require to be built.
I found a way using (can't remember the tool name) which if you loop through the imports and gives it to the tool you get the package name, then I would build wheels to have all binaries and build a container or a VM with all that's needed, thus working completely around python package managers. This was a good enough workflow for the kind of deployment we needed.
Jupyter added a layer of complexity, I deployed it alongside RStudio as browser IDEs in docker swarm. Everybody wants a different set of deps and versions, so you have to keep track of everything, and also people may use things just for development that must not be shipped to prod, so you have to keep track of that too. Also some would develop notebooks on windows and expect them to work in linux VMs/containers and even in prod.
Nowadays devs ship container images anyway through a CI so it is less of an issue. In this era docker was far from being the de-facto everywhere, some people were still afraid of this, security didn't like it, etc.
I was interested in marimo, but I became less interested when I realized that they traded off being able to assign to a variable more than once in order to allow out-of-order execution of cells.
I mean as a Jupyter user, I typically do both and just keep track of what I'm doing in my head (like a repl with many snippets I can run any time), but if I wanted to make it more predictable, I would definitely give up out-of-order execution first.
I think that’s just a fundamental tradeoff though.
Being able to run Jupyter cells independently is a feature until it’s not.
I’d say for most of my one-off work, it’s fine. But for stuff I want to share it’s not.
Yes I understand it's a tradeoff, but I'm saying I would prefer a different tradeoff. It would be more intuitive to me if running a cell always invalidated the cells below; this would make variable reassignment unambiguous, just like in a script, but still with all the visualization goodies of a notebook.
marimo dev here.
Just wanted to mention that you're always able to do this:
for _x in range(100): ...
This way, `_x` is detected as a throwaway Python variable. And it won't re-appear in other cells.
Also, within the same cell you can always re-assign. But you can't do that in another cell. We want to ensure that a variable is fully declared in one, and only one, cell.
``` # cell A, totally fine
a = 1 for _ in range(100): a = a + 1
# but don't re-assign a in another cell. ```
there are so many issues with jupyter notebooks and marimo solves none of them
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." - https://news.ycombinator.com/newsguidelines.html
It sounds like you're commenting about this from experience - if you'd like to share some of that, of course that would be welcome.
My biggest problem with Jupyter is hidden state. You have no idea what order the cells executed in and how many times to get to the current state. Pluto.jl and Marimo solve that by using reactivity to make state transparent. WYSIWYG.
I have also been thoroughly impressed how Marimo has engaged with AI agents. marimo-pair is fantastic.
What I like about Marimo notebooks over Jupyter:
1. They're just a python file so they work with python editors
2. There's no hidden state
3. I can import from them
4. I'm not accidentally committing base64 encoded image output anymore
Well, I've enjoyed it. It's easier to set up an run from a virtual environment than Jupyter notebooks. That was a big problem I had for years.
mind elaborate?