Flatpak

[Z-Man]

Starting to work on it Or rather, taking the work already done by others and claiming it for myself. See, there already is a package available on flathub, outdated and a bit broken (user configuration does not save), and flathub works as a github project with one repository for each project, and AsciiWolf already did a lot of work updating that to 0.2.9.1.0. I'm just finishing things off in this pull request.

Before that is finalized and flathub is updated, could you guys please test the builds? You find instructions how to get flatpak and flathub going on the flathub main site, and flathubbot is posting instructions on how to install the latest test build in the pull request discussion.

Things to know about flatpak: It has opinions Specifically, opinions on where applications should save their user data. Ours goes to ~/.var/app/org.armagetronad.ArmagetronAdvanced/ there. More specifically:
var data such as user.cfg goes to ~/.var/app/org.armagetronad.ArmagetronAdvanced/data/game/var
moviepacks are looked for in ~/.var/app/org.armagetronad.ArmagetronAdvanced/data/game
user configuration can go to ~/.var/app/org.armagetronad.ArmagetronAdvanced/config/game
the resource cache is in ~/.var/app/org.armagetronad.ArmagetronAdvanced/cache/resource
screenshots land in ~/.var/app/org.armagetronad.ArmagetronAdvanced/data/game/screenshot

I added the /game/ part to the paths because flatpak itself uses these directories itself and I want to avoid collisions. if you can think of better ways to distribute stuff over the data/config/cache subdirectories, I'm very open to suggestions.

The next thing to know about flatpak is that applications run in a sandbox. By default, we have access to the application specific directories above, and read only access to the game data, libraries and runtime. This is super neat! That means that if some remote code execution exploit is in our code, attackers are still trapped in the sandbox and don't get to your banking data immediately and can't encrypt your documents. They still get your saved tron passwords.
Unfortunately, that also means that debug recording and playback don't simply work. You need to give flatpak the appropriate permissions:

Code: Select all

flatpak run --filesystem=`pwd` org.armagetronad.ArmagetronAdvanced --record bug.aarec
flatpak run --filesystem=`readlink -f bug.aarec`:ro org.armagetronad.ArmagetronAdvanced --playback bug.aarec

Uhh, and I guess the first thing you would have needed to know is that flatpak as a whole really takes up a lot of hard drive space, a bit like Steam. If you can't spare aroud a gigabyte, don't start.

[Z-Man]

Flathub staff is taking their time to update the package, and my request mail is stuck in moderation limbo. It's probably the time of the year, I'll give them a week or so, then pester them directly

Flathub isn't going to fulfill all our needs anyway; on the main hub, they only want stable releases, and even on the beta, they don't want nighties, only things that got some testing already. So I extended our build system. It's a bit of a different setup than all the rest. All the other builds are done in docker containers controlled by the main source module and only that. That does not work here: For one, Flatpak build definitions should contain direct references to source tarballs with sha checksums, and of course we only know those after the tarball is made. So they can't be checked in to the main source. Like the download site and the Zero Install streams, the Flatpak definitions need to be in a separate git repository. The second difficulty is that Flatpak builds don't work inside rootless docker. I experimented a bit and ended up with another user on my build server with another github runner, this time a shell runner, attached to yet another project that contains the Flatpak build scripts. Flatpak builds themselves are also sandboxed, so having a shell runner for this one small purpose is not much of a security risk. And the SDK is super easy to set up.
So the process is now this: Main builds run. In the deployment stage, the corresponding branch of the Flatpak definition project is updated. That update sends a message to the build project that has a very small CI process that fetches the definitions and builds them.

The result is our very own little Flatpak repository with all of our builds (except for the occasional one that will turn out to break the Flatpak build).

I have to say... I kind of like Flatpak now. I was a little scared; see, you have to add all libraries that your project needs, and that's best done by compiling them from source. The ones 0.2.9 uses are all either in the runtime or covered by common installation instructions one can use. For 0.4, ftgl, protobuf and boost are not. I was very worried about boost. And indeed, it is the most complicated one of the three. Look at this monstrous build definition!
Wait... 666 bytes? 25 lines? Half of them copy/pasted instructions on what to clean up?
Yeah. Turns out you just tell it what file to download and in this case, which commands to execute inside the unpacked archive. For the other two, you just point it to the download. Flatpak knows on its own how to handle the configure/make/make install dance. Adapting the definition for 0.4 took just two hours. The liberating thing here is that you're building directly against an up-to-date runtime. What the configure scripts find is safe to use. The Steam runtime, for comparison, is outdated; we need to pull in libraries from later Ubuntu versions and include them in the package. For Zero Install, AppImage and Itch.io, we need to very carefully build some things from source with very carefully disabled configuration options so we don't have to include a ton of extra libraries we're not even using. Luckily, it's the same process for all three. And Debian... I admit, I don't understand the first thing about how that works.

So yeah, if I had to restrict myself to ONE binary distribution system per platform with total disregard for the users, they'd be Itch for Windows, Flatpak for Linux clients and... blast. I guess docker for the servers. Is that allowed? Right, I make the hypothetical rules, so yes, it is.

Now that the entry hard drive space fee is paid, I'm switching some programs from AUR to Flatpak. Zoom, Discord and Godot go first.

[sinewav]

Any issues with performance? I've read that some applications such as multimedia editors (video NLE, audio/DAW) can have major performance issues. I've got three Flatpak programs on my system and so far no complaints.

[Z-Man]

A good question! I had not considered that. Yes, sandboxing is bound to cost some performance. They made special arrangements for OpenGL, it doesn't need to go through some bottleneck and apps have more or less direct access (noticeable by the user because Flatpak installs a graphics driver specific package to each runtime). I did some quick benchmarks with a ladle recording in a small window so GPU fillrate was no factor. Best result from Flatpak from three runs:

Code: Select all

[[email protected] ladle120]$ flatpak run --file-forwarding --branch=stable org.armagetronad.ArmagetronAdvanced --playback @@ round1_z_txa_vs_km.aarec @@ --benchmark
201926 frames in 90.1726 seconds: 2239.33 fps.

AppImage, Zero Install and a self-built binary are about equal, their worst and best results were from AppImage:

Code: Select all

$ ~/Games/ArmagetronAdvanced-0.2.9.1.0.AppImage --playback round1_z_txa_vs_km.aarec --benchmark
201926 frames in 87.1984 seconds: 2315.71 fps.
[[email protected] ladle120]$ ~/Games/ArmagetronAdvanced-0.2.9.1.0.AppImage --playback round1_z_txa_vs_km.aarec --benchmark
201926 frames in 84.047 seconds: 2402.54 fps.

So there is a measurable performance hit in the 5-10% range. That's not a number that would worry me. Steam is harder to measure because you don't get to see the game's console output, but also seems to see a similar drop. Runtime of that recording measured by hand was 91 seconds, twice. It would be interesting to compare that with a manually set up sandbox run on the other systems. I was meaning to set up a firejail profile for that.

Startup times also show some overhead. Our systems from fastest to slowest:

Code: Select all

# Native:
$ time ./armagetronad --version
This is armagetronad version 0.2.9.2_alpha_z2427.
real	0m0,015s

# Zero Install
$ time 0launch -c 'https://0install.armagetronad.org/armagetronad-stable.xml' --version
This is Armagetron version 0.2.9.1.0.
real	0m0,031s

# AppImage
$ time ~/Games/ArmagetronAdvanced-0.2.9.1.0.AppImage --version
This is Armagetron version 0.2.9.1.0.
real	0m0,056s

# Flatpak
$ time flatpak run --branch=stable org.armagetronad.ArmagetronAdvanced --version
This is Armagetron version 0.2.9.1.0.
real	0m0,071s

Interesting. I thought AppImage would come in last because that has to mount an entire filesystem, but Flatpak is still a bit slower. Of course, we're talking a mere 56 ms overhead compared to the native binary, that's not anything one would even notice.

[Z-Man]

Huh, Steam for Linux is moving into the the same direction, technically, here's a talk about it:
https://archive.fosdem.org/2020/schedul ... ers_steam/
They're trying to solve two problems; first, the current method they are using, setting an elaborate LD_LIBRARY_PATH to their runtime, is bound to break at some point. And they can't update the library content in the corrent model. So they borrow the sandbox from Flatpak, using their runtime the same way Flatpak is using theirs, and then give the game developers a choice which runtime to build and run with. That choice bit doesn't work yet Not even with the beta steam client, no matter what you do, you still get the old runtime, so the test build I did for the updated runtime would not run.

In actual Flatpak news, I'm now in control of the Flathub presence. Yay!

[sinewav]

Flathub presence

Speaking of presence, the main website needs attention, badly. With all the different ways people can now get Armagetron we don't exactly have a welcome mat laid out for everyone visiting armagetronad.net. The last redesign was in 2005 (Lackadaisical). How about getting a committee together to address this? I may have a lot of free time coming up so I would be glad to help the modernization efforts.

[Z-Man]

The last time this came up we had a committee, that didn't work out too well; there were good suggestions, but no decision. Just putting someone in charge sounds like a better idea.If you're interested and have time, feel free to just grab the position No pressure, no hurry.
I could deal with the technicalities; a server is there and those frameworks are ridiculously easy to set up these days. I myself am quite fond of the idea of static site generators, of which there are at least half a dozen decent ones. I'm using Jekyll for the download site. Works quite well, but the templating language is very, very verbose. Like, here's a comment in it:

Code: Select all

{%- comment -%}Keep tabs on the number of reserved slots per news category{%- endcomment -%}

I obviously haven't figured out how to make it beautiful But I'm not married to the concept, if Wordpress or whatever gives better results or is easier to use or customize, fine by me. I only have to paste in release notes and can get them in practically any format out of the build system.

[Monkey]

Flatpack...flathub...appimage...zeroinstall...docker...
You forgot to mention systemd for a full house.

Thank **** I left Linux when I did.

[Z-Man]

Snap would be the more fitting completion, systemd has nothing to do with us!
(Not touching snap myself. Wouldn't mind if somebody else did, but my own experiences with it were not so great. Snaps polluting my 'du' output. And the first app Ubuntu only gave me via snap when it was perfectly available as a .deb before, Skype, absolutely did not work. And yeah, a central, proprietary repository is hard to defend in an OS world.)