On Debian-based distros there are several tools.

squid-deb-proxy can do most of this - it fetches and locally caches lists and packages. Clients install squid-deb-proxy-client which uses multicast-DNS to discover the local proxy.

Packages are fetched and cached by the proxy first time they're requested and thereafter served locally (subject to lifetime/space etc. squid configuration).

There are ways to pre-populate the cache but you're always going to have situations where package updates have to pull from remote archives.

apt-mirror is designed for a similar purpose. It allows creating a sub-set or entire mirror of one or more upstream archives.

Clients can then be pointed at the local mirror which could be on the same host or LAN.

Generally, 'packages' on github will be source-code only so you'd need a quite complex build server, or at least per-package specific links in your local cache rules for pre-built binaries.

> it's been driving me nuts that EVERYTHING I do needs to be apt-gotten off the internet,

One solution is to do it once only: grab the base image, install all the packages you need, and then make an image of it so you don't start from scratch again next time.

>the base image doesn't even include basics like screen/tmux.

Easily arguable that RPi isn't made with such use cases in mind. I'm all in favor of removing unnecessary bloat if only a small percentage of user base is ever going use it, especially when the software is readily available.

>If those repos are inaccessable for any reason, I have a bunch of hardware that's very hard to do anything useful with.

How so? They are just Linux boxes, you can just download the source code and compile the binaries you need. Pre-built packages are not necessary for functional OS.

NixOS solves all of these problems.

a) define your entire system state and dependencies in a single, declarative file

b) prebake an image based on this file

It's all done and ready, available now, no hacking around required.

Yes, software accessibility is an issue in the most pronounced disaster scenarios. But anybody who looks at a device of such exquisite craftsmanship, which is evidently already set up to function, and frets about downloading software after some apocalyptic scenario has missed at least half the point.

Since these are small devices, I think a minimal image is a necessity. What you will use is not what someone else will use.

As to caching, I tried setting one up and found out a few things.

I originally tried apt-cacher-ng:

https://www.unix-ag.uni-kl.de/~bloch/acng/

but had trouble (can't remember exactly what) and muddled through polipo instad.

I found out:

- it was a lot easier and faster than copying cached packages around

- when doing "apt-get update; apt-get upgrade" the cache really sped up multi-machine updates. The first machine was slow, the rest were fast.

- By far, the majority of cache traffic was for updates to base stuff. foo-1.2, foo-1.2.1 foo-1.2.2, etc.

- second was prerequisites for packages I needed.

- the few specialized packages I used were not updated as frequently.

You may want to look at debos for pre-baking debian/ubuntu images.

https://github.com/go-debos/debos

Yep, big problem with the repo model that Linux insists on using for everything. Offline is a 5th class citizen because of course everyone lives in SV or a university dorm.

FWIW providing you have the kernel and firmware basically everything can be built from source.

I've built most of the base repository of Arch Linux on my RPi4 without cross compiling. I'm buying a few more so that I can go 'full Gentoo' and pretty much rebuild the world.

Have a recording of a container booting from a rebuilt userland: https://asciinema.org/a/283303

It might feel like the ARM packages are somehow special but really only the blobby bits like the RPi firmware is. Everything else is just your bog standard armv7/aarch64 ELF.

So yeah, back up a GCC binary or bootstrap, I guess? I can probably email you a tmux binary in a pinch, plus I have a full local mirror of the repo for the apocalypse? :P

Debian provide a physical artefact containing all the packages that could be used without internet access. I don't know whether this also works for Raspian: https://www.debian.org/CD/

This is why projects like yocto and buildroot exist.

They help you maintain a stable software stack over time. And generate a fully contained software image that can be flashed without needing the internet afterwards. (All downloads happen during compilation)

How often are the repos inaccessible? I've come around to the opinion that, unless you have a very specific use case, networking is an essential element of any (Linux) computer system. Once you have that, you can benefit from a world of free software, with updates all handled automatically.

Personally I find it almost magical that I can install and update almost any software I need with a short command.

You can build your own image/rootfs

Imagine that you can build your rootfs is a docker container and export it to IMG

I don't think I've ever seen the rpi apt repository offline, so I'm not sure why you're so worried. Embedded computers tend towards lean rather than heavy and that is the mindset you're going to encounter.

Can't you rsync down your own repository?

Most of the important stuff is in busybox and gcc (maybe Python.) Cache those and their dependancies and you have most of what you want.