I didn't really get what "a command-line Web, Gemini, and Gopher browser that allows you to work offline" meant, like how is it offline? So I think this is the bit of info I was missing:
Every content you visit is cached and can be visited later while offline. If you try to visit a content not available in your cache, it will be marked to be downloaded later. Offpunk allows you to synchronise you computer once every hour, day or week and work offline without being interrupted.
Then, I ran "offpunk" and got a new terminal prefixed "ON>".
Curious, I entered "help" and got a list of topics. A few stood out as immediately interesting, so I tried "open https://benovermyer.com" to open my site. It said I needed to "go" somewhere first. So, I tried "go https://benovermyer.com" instead.
That showed me a text preview of my front page, interestingly with content between headers stripped out. OK, that's neat, but now what?
So I now tried "open", and this time it opened the page in my browser, Vivaldi. It was my site, sure enough, but cached locally and missing my main image. All the CSS was intact, though.
That's as far as I've gotten at this point, but it was an interesting enough experience that I thought I would share. It's a very different flow from what I was expecting.
So you go to a site. If the site is not complete, try "v full" (for "view full").
The reason is that there was no unmerdify rule for your site and that readability mistakenly assumes that lot of your site is spam/ads (readability doing that is the reason why we push for unmerdify)
> Something wonderful happened on the road leading to 3.0: Offpunk became a true cooperative effort. Offpunk 3.0 is probably the first release that contains code I didn’t review line-by-line.
I felt true dread reading a sentence like this. I had to reread to make sure the author means there are other trusted contributors now.
As it is often the case these days, some projects are quite proud of announcing that no human has written or reviewed their code.
But, trully, there’s no much to search. The beauty of Gemini is how small it is: all technical informations should be there : https://geminiprotocol.net/
To be fair, "Gemini" is one of those overused words for projects and it was unsearchable before Google. It's in the same category as Atlas, Nexus and Styx.
It's as if someone took a look at Gopher and tried to do it in 21st century.
It's not bad, but the (intentional) lack of user-input takes the whole experiment back. The only user input is a small search query. It's intentionally set up against building any kind of BBS/Forum. So you need a different way to write and upload, which - in my opinion - kills the whole project. YMMV.
Lagrange - the most used browser - used to have something called "Titan" as user upload but that was never specified and nobody used it and I think it's removed from Lagrange now.
edit: ah it still supports it (Titan). It just seems there is no server in the world that uses it.
If I have a criticism of the protocol it is that gemtext errs too far on the side of simplicity, particularly in not allowing inline links or formatting. I would have preferred if it was closer to markdown.
The limited input support doesn't bother me. I don't see the need for social media through Gemini when there are loads of other protocols we can use for that. And even the limited form of input that is supported lets you do a lot if you get creative with it (in fact there are a few social media-like Gemini pages out there).
You can open several incignito browser tabs to try it out side by side. It's easy to compare!
Note: In the US, doing English searches, using a chrome-based browser. Coming in with Firefox WILL give you different search results. Sometimes I do that intentionally (can be useful).
This is really interesting, is there a way to provide downloadable caches for websites where that would be legal? I could Imagine just pre-downloading wikipedia, stack overflow, all kinds of documentation, etc. In a compressed and preorganized format instead of scraping it every time.
I tried to add kiwix support to Offpunk in the early days but kiwix do not support incremental update of the wikipedia database, which makes it a huge pain point.
Yeah, that’s the biggest disadvantage of kiwix. It probably makes their hosting bill enormous as well. Zimdiff/zimpatch was implemented a decade ago so I’m not sure what the holdup is.
I wonder when contemporary developers will (re)invent Emacs/Gnus: the unified inbox for email, feeds, and news, because what really matters are text messages + eventual multimedia content, personal and private scoring to manage them, and a consistent local UI that allows for personal archiving and resharing.
I've looked at the Fediverse, objectively with little hope and many design issues, I'm watching Nostr with interest even though it seems more like a rough sketch lacking the ideas to move forward, but that concept of Gnus and Usenet, so simple in itself, still hasn't managed to resurface.
- offpunk is an offline content browser/reader. Main component is fetching/caching/displaying ressources
- offpunk is developed as a set of components that can be used separately (openk, ansicat, netcache)
- offpunk delegates as much as possible to other UNIX tools (less for browsing/reading, chafa for images, grep to find in a page, $EDITOR for editing needs )
- offpunk is pure CLI tool. You type commands, results is displayed in your terminal or in less. There’s no "keyboard shortcuts" or "environment". It is a prompt on which you type commands
- There’s no "configuration" in offpunk. The only (but powerful) way on configuring is having offpunk launch commands at starts (commands listed in offpunkrc). So no "configuration language" or syntax or plugins or whatever.
- last but not least: basic use of Offpunk is simple. You are not required to learn much and you use only what you want. Lot of Offpunks users don’t use the Web/HTTP part and use it as a straight Gemini browser (for the record, Offpunk is a fork of AV-98, the very first Gemini browser)
My intention wasn't to compare software but rather paradigms; Gnus is only relevant here because it unifies email, feeds and news into a single UI. In other words, the human user sees everything as generic "posts" an approach not unlike many Nostr clients, to name something modern. It's the paradigm of saying "in the end, what matters is the message". As long as there's a readable amount of them, you don't need anything else; when they reach a certain volume, well, you need to be able to "filter" them somehow so that some are never seen/read, for others you only see the title at a glance, and some are actually read. This is the principle of scoring, which is even older, I think it was part of the first PARC Altos.
What I meant is that I wonder how long it will take nowadays to go back to creating a decentralized model or, since overhead allows for it today, a distributed one, that serves modern forms of human communication:
- blogs (e.g. Nostr's long-form notes in Habla, or WireFreely for the Fediverse)
- non-synchronous short messages (e.g. Twitter/X style)
- synchronous short messages, i.e. chat
With a decentralized/distributed network for distribution where everyone keeps what they want on their own hardware.
On the sidelines, it would be nice today to also see synchronous audio and audio+video, meaning calls and conferences, all in a single UI and with at most two or three protocols on the network side (one for asynchronous messages and media, one for chat if the asynchronous one doesn't cut it, and one for calls).
Without the end user having to make personal collages if they don't want to, using an app that is go-installable, pip-able, cargo-build-able, basically something that both those who want to try it and distro packagers can add quickly. This would help spread something among techies/nerds/geeks and also works for the end user, who would be introduced to this solution by the techies/nerds/geeks. To me, this is what's missing to see the big platforms currently in fashion get toppled.
Seeing projects like Offpunk inspired the thoughts above; that was the point :)
well, indeed. If you use mutt and use the "reply" feature in Offpunk, you will see how well emails and blog/gemini posts merge well ;-) (this will be the subject of another post later)
As a long time Emacs user, I never even tried Gnus, or used it as a calendar (except for some time tracking in org-mode). How would calendar invites work there? How well does it support shared calendars to determine busy/free information of others?
Gnus it's dog slow, be with email or with usenet. I say this as an ex-Emacs user where I even plugged slrn's cache in order to speed up things, but over 100k messages that was unusable in my netbook, even under 64 bit machines and native compilation. Slrn did it better. On RSS, I use sfeed which is more Unix like and I just plumb lynx/links or whatever I like as a reader. And fast, much fast than GNUs, Elfeed or the core RSS reader in Emacs.
OTOH, Emacs it's the only libre Usenet reader for Android. Go figure, and that being a dead simple protocol. Despite of that, lots of Thunderbird forks in FDroid didn't adapt the Usenet part yet.
Offpunk it's slow but adding multiprocessing with flock (for python3 maybe) would be a piece of cake in order to allow parallel downloads while syncinc.
I don't think Gnus is that bad once you spend some time setting it up. For groups with a ton of content where I mostly want to search, I found it was better to just download the whole group and index into notmuch. I could query 20 years of the Smalltalk USENET group or the Supercollider mailing list instantly.
each tool needs very specific set of options so we can’t just let the user input any command. And offpunk has been built for use with chafa since the start. Timg is just an historical workaround were Chafa had some bugs.
But there are currently some discussions about that on the mailing-list.
I didn't really get what "a command-line Web, Gemini, and Gopher browser that allows you to work offline" meant, like how is it offline? So I think this is the bit of info I was missing:
Every content you visit is cached and can be visited later while offline. If you try to visit a content not available in your cache, it will be marked to be downloaded later. Offpunk allows you to synchronise you computer once every hour, day or week and work offline without being interrupted.
(from: https://offpunk.net/whatisoffpunk.html)
So its kindof designed to be offline with occasional sync. Interesting.
I was curious, so I tried it out.
I ran `apt install offpunk` on my Debian laptop.
Then, I ran "offpunk" and got a new terminal prefixed "ON>".
Curious, I entered "help" and got a list of topics. A few stood out as immediately interesting, so I tried "open https://benovermyer.com" to open my site. It said I needed to "go" somewhere first. So, I tried "go https://benovermyer.com" instead.
That showed me a text preview of my front page, interestingly with content between headers stripped out. OK, that's neat, but now what?
So I now tried "open", and this time it opened the page in my browser, Vivaldi. It was my site, sure enough, but cached locally and missing my main image. All the CSS was intact, though.
That's as far as I've gotten at this point, but it was an interesting enough experience that I thought I would share. It's a very different flow from what I was expecting.
the "open" open outside.
So you go to a site. If the site is not complete, try "v full" (for "view full").
The reason is that there was no unmerdify rule for your site and that readability mistakenly assumes that lot of your site is spam/ads (readability doing that is the reason why we push for unmerdify)
> Something wonderful happened on the road leading to 3.0: Offpunk became a true cooperative effort. Offpunk 3.0 is probably the first release that contains code I didn’t review line-by-line.
I felt true dread reading a sentence like this. I had to reread to make sure the author means there are other trusted contributors now.
As it is often the case these days, some projects are quite proud of announcing that no human has written or reviewed their code.
OP here: Didn’t even come to my mind while writing this. This would be totally against the philosophy of the projects.
Don’t worry: I wanted to celebrate humans cooperating over good old emails to write code in their editor without any help ;-)
Thank you for confirming, and congrats for shipping!
This looks interesting! I wanted to find more info about the Gemini protocol, and it's annoying how un-searchable it is now due to Google Gemini.
Try "gemini protocol", it helps a lot.
But, trully, there’s no much to search. The beauty of Gemini is how small it is: all technical informations should be there : https://geminiprotocol.net/
To be fair, "Gemini" is one of those overused words for projects and it was unsearchable before Google. It's in the same category as Atlas, Nexus and Styx.
It's as if someone took a look at Gopher and tried to do it in 21st century.
It's not bad, but the (intentional) lack of user-input takes the whole experiment back. The only user input is a small search query. It's intentionally set up against building any kind of BBS/Forum. So you need a different way to write and upload, which - in my opinion - kills the whole project. YMMV.
Lagrange - the most used browser - used to have something called "Titan" as user upload but that was never specified and nobody used it and I think it's removed from Lagrange now.
edit: ah it still supports it (Titan). It just seems there is no server in the world that uses it.
If I have a criticism of the protocol it is that gemtext errs too far on the side of simplicity, particularly in not allowing inline links or formatting. I would have preferred if it was closer to markdown.
The limited input support doesn't bother me. I don't see the need for social media through Gemini when there are loads of other protocols we can use for that. And even the limited form of input that is supported lets you do a lot if you get creative with it (in fact there are a few social media-like Gemini pages out there).
It's interesting how long we're been using web search, and the total lack of user education on the features.
"Gemini protocol" with quotes should help.
+Gemini +protocol for more fuzzy, less accurate results
these operators straight up do not work anymore in any mainstream search engine.
I use them daily with Google. Works great.
You can open several incignito browser tabs to try it out side by side. It's easy to compare!
Note: In the US, doing English searches, using a chrome-based browser. Coming in with Firefox WILL give you different search results. Sometimes I do that intentionally (can be useful).
This is really interesting, is there a way to provide downloadable caches for websites where that would be legal? I could Imagine just pre-downloading wikipedia, stack overflow, all kinds of documentation, etc. In a compressed and preorganized format instead of scraping it every time.
The cache is pure and straight files-in-folders. This makes it trivial to browse the cache by hand:
cd .cache/offpunk/https/news.ycombinator.com/item cat "id=46943752"
So it could be trivially shared.
The "netcache" tool gives you the cached content or, with --path, returns the path were to find the contentd.
The only point is to preserve the file-modification attribute, which serves to know the age of a cached ressource.
Kiwix basically provides this. You can use kiwix-serve to serve the downloaded zim files locally.
I tried to add kiwix support to Offpunk in the early days but kiwix do not support incremental update of the wikipedia database, which makes it a huge pain point.
Yeah, that’s the biggest disadvantage of kiwix. It probably makes their hosting bill enormous as well. Zimdiff/zimpatch was implemented a decade ago so I’m not sure what the holdup is.
OFFPUNK sounds like a good name for a punk-like movement to rebel against the entire internet and live offline.
see that contributed picture: https://offpunk.net/screenshots/resist.png
I feel like that movement already exists but uses a different name. Or doesn't use a name at all.
I wonder when contemporary developers will (re)invent Emacs/Gnus: the unified inbox for email, feeds, and news, because what really matters are text messages + eventual multimedia content, personal and private scoring to manage them, and a consistent local UI that allows for personal archiving and resharing.
I've looked at the Fediverse, objectively with little hope and many design issues, I'm watching Nostr with interest even though it seems more like a rough sketch lacking the ideas to move forward, but that concept of Gnus and Usenet, so simple in itself, still hasn't managed to resurface.
OP here and Vim/UNIX fan.
I get the idea behind "reinventing Emacs".
But there are main differences:
- offpunk is an offline content browser/reader. Main component is fetching/caching/displaying ressources
- offpunk is developed as a set of components that can be used separately (openk, ansicat, netcache)
- offpunk delegates as much as possible to other UNIX tools (less for browsing/reading, chafa for images, grep to find in a page, $EDITOR for editing needs )
- offpunk is pure CLI tool. You type commands, results is displayed in your terminal or in less. There’s no "keyboard shortcuts" or "environment". It is a prompt on which you type commands
- There’s no "configuration" in offpunk. The only (but powerful) way on configuring is having offpunk launch commands at starts (commands listed in offpunkrc). So no "configuration language" or syntax or plugins or whatever.
- last but not least: basic use of Offpunk is simple. You are not required to learn much and you use only what you want. Lot of Offpunks users don’t use the Web/HTTP part and use it as a straight Gemini browser (for the record, Offpunk is a fork of AV-98, the very first Gemini browser)
My intention wasn't to compare software but rather paradigms; Gnus is only relevant here because it unifies email, feeds and news into a single UI. In other words, the human user sees everything as generic "posts" an approach not unlike many Nostr clients, to name something modern. It's the paradigm of saying "in the end, what matters is the message". As long as there's a readable amount of them, you don't need anything else; when they reach a certain volume, well, you need to be able to "filter" them somehow so that some are never seen/read, for others you only see the title at a glance, and some are actually read. This is the principle of scoring, which is even older, I think it was part of the first PARC Altos.
What I meant is that I wonder how long it will take nowadays to go back to creating a decentralized model or, since overhead allows for it today, a distributed one, that serves modern forms of human communication:
- blogs (e.g. Nostr's long-form notes in Habla, or WireFreely for the Fediverse)
- non-synchronous short messages (e.g. Twitter/X style)
- synchronous short messages, i.e. chat
With a decentralized/distributed network for distribution where everyone keeps what they want on their own hardware.
On the sidelines, it would be nice today to also see synchronous audio and audio+video, meaning calls and conferences, all in a single UI and with at most two or three protocols on the network side (one for asynchronous messages and media, one for chat if the asynchronous one doesn't cut it, and one for calls).
Without the end user having to make personal collages if they don't want to, using an app that is go-installable, pip-able, cargo-build-able, basically something that both those who want to try it and distro packagers can add quickly. This would help spread something among techies/nerds/geeks and also works for the end user, who would be introduced to this solution by the techies/nerds/geeks. To me, this is what's missing to see the big platforms currently in fashion get toppled.
Seeing projects like Offpunk inspired the thoughts above; that was the point :)
well, indeed. If you use mutt and use the "reply" feature in Offpunk, you will see how well emails and blog/gemini posts merge well ;-) (this will be the subject of another post later)
As a long time Emacs user, I never even tried Gnus, or used it as a calendar (except for some time tracking in org-mode). How would calendar invites work there? How well does it support shared calendars to determine busy/free information of others?
There is https://github.com/emacs-mirror/emacs/blob/master/lisp/calen... and https://github.com/dengste/org-caldav but that not much related to Gnus approach of "unifying messages in a single UI", it could be used to import from ics files to org-agenda or creating an ics file or sync org-agenda with Radicale/Baïkal/Davis CalDAV servers
Gnus it's dog slow, be with email or with usenet. I say this as an ex-Emacs user where I even plugged slrn's cache in order to speed up things, but over 100k messages that was unusable in my netbook, even under 64 bit machines and native compilation. Slrn did it better. On RSS, I use sfeed which is more Unix like and I just plumb lynx/links or whatever I like as a reader. And fast, much fast than GNUs, Elfeed or the core RSS reader in Emacs.
OTOH, Emacs it's the only libre Usenet reader for Android. Go figure, and that being a dead simple protocol. Despite of that, lots of Thunderbird forks in FDroid didn't adapt the Usenet part yet.
Offpunk it's slow but adding multiprocessing with flock (for python3 maybe) would be a piece of cake in order to allow parallel downloads while syncinc.
I don't think Gnus is that bad once you spend some time setting it up. For groups with a ton of content where I mostly want to search, I found it was better to just download the whole group and index into notmuch. I could query 20 years of the Smalltalk USENET group or the Supercollider mailing list instantly.
offpunk’s slowness has two main sources (while offline):
- loading modules at startup (will be solved in 3.1 with lazy loading, patch is pending)
- parsing HTML with lot of pictures (because we wait for chafa for each picture)
I’m not sure how multiprocessing could really help that much but I would be interested.
While online, sure, the blocking http calls are something that will be parallelized in the future
syncing URL lists should be done in parallel. Something like xargs+flock but under Python.
Contributions are welcome. I even believe that this whole code should be migrated from python-requests to curl.
There should ve a variable to choose your inline image reader, between chafa, timg, some wrapper around w3mimgdisplay...
Harcoding it it's bad.
each tool needs very specific set of options so we can’t just let the user input any command. And offpunk has been built for use with chafa since the start. Timg is just an historical workaround were Chafa had some bugs.
But there are currently some discussions about that on the mailing-list.