And now we have git.

What do you want to see in Armagetron soon? Any new feature ideas? Let's ponder these ground breaking ideas...
User avatar
Z-Man
God & Project Admin
Posts: 11585
Joined: Sun Jan 23, 2005 6:01 pm
Location: Cologne
Contact:

And now we have git.

Post by Z-Man »

Surprise!

https://gitlab.com/armagetronad

Main repository with the game code:
https://gitlab.com/armagetronad/armagetronad

Using the excellent git-bzr, which makes bzr branches read/writable from git, I set up a bidirectional sync script that runs on a webhook on pushes to either side (https://code.launchpad.net/~z-man/armag ... zr-scripts), small commits are on the other side in about a minute. We can, for the time being, freely pick between working on the git or bzr side.

That is, as long as we avoid the small set of git features on these branches that bzr simply does not support, namely:
-submodules
-large file system

There are two branches in each repository; 'master' is equivalent to the 0.4 family on the bzr side, 'legacy_0.2.8' matches the 0.2.8 family. They're properly related, of course.

Reason for choices made in the next post.
User avatar
Light
Reverse Outside Corner Grinder
Posts: 1667
Joined: Thu Oct 20, 2011 2:11 pm

Re: And now we have git.

Post by Light »

Great! I was going to suggest this in one of your previous threads. I never remember how exactly to use bzr because in normal life I never find people using it. Git brings us to the present at least and makes for easy to handle and merge branches.

Good move. I wouldn't mind seeing the bzr become history.
User avatar
Z-Man
God & Project Admin
Posts: 11585
Joined: Sun Jan 23, 2005 6:01 pm
Location: Cologne
Contact:

Re: And now we have git.

Post by Z-Man »

The main repository is also mirrored on launchpad, I suppose as a read only clone:
https://code.launchpad.net/~armagetrona ... magetronad

The other helper repositories are not mirrored to launchpad because when I tried, it insisted on also naming them just 'armagetronad', not ending well.

Decisions:
Why expand to git?
We've partially been through this here, as well. First, I really like bzr. I personally do not care that its development has stopped, it has all the features I think a source control system needs. There is only one thing that annoys me about it; if you do pull/work/commit/push and somebody else pushed while you were working, to keep what bzr considers the mainline of development intact, you need to do a merge in a separate clone of the branch or do some uncommit shuffling. Since git does not care about a mainline and merges are symmetrical, there you can just do merge/push and be done. Or rebase/push. (*)
Key advantages of git, though:
- larger userbase. It's likely any new recruits already know git and hang out at one of the git hosting places. Also, the ecosystem around git nowadays is way richer than for bzr. An argument from popularity, certainly, but these are real advantages and not just doing what all the cool kids are doing.
- supports even more workflows. Quick and cheap local branch creation is quite something.
- rebasing. 'rebase -i' is very useful if you want to make yourself appear smarter and more organized than you actually are, it allows you to completely rewrite your unpublished commit sequence.

Two things git annoys me with:
- it's easy to forget which branch you have checked out currently in which repository and do something silly by mistake. this thing helps, altering your prompt to show you which branch you are on.
- git does not remember which branch you merged from. In bzr, I now can have a 0.4 branch and in it, if I type 'bzr merge', always merge from 0.2.8. In git, I would usually do 'git merge origin/legacy_0.2.8'. That's a maintainer problem, though, when doing regular work, I guess it's not too common to merge from the same branch the whole time.

I don't really know which of the two is better for merging. We'll have to see. The 0.2.8 -> 0.4 merge situation I mostly deal with is, I hope, not something typical anyway.

Why not use launchpad's git hosting?
It's very barebones, not even counting the already mentioned problem with the mirrored repositories. You can't set it up so that force pushes are disabled, it seems. Also, I wanted to look around a bit, and Launchpad is constantly annoying me with internal timeouts, forcing me to press page refresh.

Why not github?
I haven't said yet we're not going there :) In fact, I do think we should have at least mirrors up there for visibility and again, argument from popularity. But I went for gitlab first because
- it's open source
- it allows you to export your data
- both mean that if need be, one can switch to self-hosting
- it has powerful CI/CD tools and even provides servers to run them on

For those too afraid to ask: CI stands for continuous integration and usually means that taks such as automated tests are run on every commit. CD stands for continuous delivery/deployment and typically means that you make it extremely easy to trigger a fully automated build and release process, for example by triggering a build for every commit to a specific branch. In our case, full CD would mean automatically delivering the client to the players. Especially for Windows, a couple of puzzle pieces are missing, Steam being a candidate for the delivery of the client.
Launchpad also has CI/CD tools, but only targetting the Ubuntuverse, ppa and Snap. Useful, certainly, and they look easy to configure, so I'll play with those; but they won't get us all the way.
The power of these things is that properly set up CI can make you very certain the quality of the checked in work is good, and CD makes it not a big deal if something bad survives the release process anyway because you then just commit your fix and press the green button again. And it allows you to release often and establish a tight feedback loop with the players. If you've been playing buzzword bingo, GAMES AS A SERVICE.

For me not so relevant because I think there is a reasonable chance they actually are good guys now, but maybe an additional reason for a couple of you: gitlab is not owned by Microsoft.

(*) We probably should set a rule there. Suggestion: our only rule regarding rebase should be, do not rebase anything already pushed to the main repositories or where you know others pulled from it. Common sense, really. I would encourage using 'rebase -i' to clean up messy local histories, especially to squash commits that have bugs you already fixed, but that's all.
User avatar
Light
Reverse Outside Corner Grinder
Posts: 1667
Joined: Thu Oct 20, 2011 2:11 pm

Re: And now we have git.

Post by Light »

I would vote GitHub sort of for popularity. Most projects I work with or put issues on use GitHub, so one account manages them all. I don't use anything on gitlab, except a self hosted version of it, so I don't even have an account there. I think the most popular is good for us making easier to recruit developers.
User avatar
aP|Nelg
Match Winner
Posts: 621
Joined: Wed Oct 22, 2014 10:22 pm
Contact:

Re: And now we have git.

Post by aP|Nelg »

Light wrote:I would vote GitHub sort of for popularity. Most projects I work with or put issues on use GitHub, so one account manages them all. I don't use anything on gitlab, except a self hosted version of it, so I don't even have an account there. I think the most popular is good for us making easier to recruit developers.
I would say have them both on Github and Gitlab then. It does add another site to sync with, though.
User avatar
Light
Reverse Outside Corner Grinder
Posts: 1667
Joined: Thu Oct 20, 2011 2:11 pm

Re: And now we have git.

Post by Light »

aP|Nelg wrote:It does add another site to sync with, though.
Doesn't seem to bad as long as there's like one commit a month (might be being generous). I'm unsure what's going to happen to all of this syncing once we hit a conflict on one of them though.
User avatar
Light
Reverse Outside Corner Grinder
Posts: 1667
Joined: Thu Oct 20, 2011 2:11 pm

Re: And now we have git.

Post by Light »

Could we ignore eclipse files as well? (attached patch - using .txt because .patch isn't allowed)
Attachments
eclipse-ignore.txt
(226 Bytes) Downloaded 292 times
User avatar
Z-Man
God & Project Admin
Posts: 11585
Joined: Sun Jan 23, 2005 6:01 pm
Location: Cologne
Contact:

Re: And now we have git.

Post by Z-Man »

Sure, can do.

Syncing another repository, especially git-git, isn't going to cause issues anytime soon. Two people not only need to commit concurrently in a one minute time window, their commits also need to be in conflicts with each other. Non-conflicting concurrent commits are auto-merged. That worked for bzr/svn where the time window was much larger, it's even less problematic here. I should get to setting up something on github maybe Sunday or sometime next week.

But you know you can use your github login to sign into gitlab, right? :) You'd have to tell gitlab your ssh key if you want to push code, but that's all.

Is Eclipse good now for C++ code? It was dreadfully sluggish back when I tried it and there wasn't much C++ support, but it was when we were still on CVS.
User avatar
Light
Reverse Outside Corner Grinder
Posts: 1667
Joined: Thu Oct 20, 2011 2:11 pm

Re: And now we have git.

Post by Light »

I just signed up to GitLab with my Google login, so no big deal really. It probably isn't worth doing both.

Also, I code on a gaming laptop. I use my personal Alienware 17r4 for work and while at home. Eclipse runs great for me. I can't say whether or not it works well on low end computers.
User avatar
Z-Man
God & Project Admin
Posts: 11585
Joined: Sun Jan 23, 2005 6:01 pm
Location: Cologne
Contact:

Re: And now we have git.

Post by Z-Man »

No worries about low end computers, we could never make an IDE mandatory for building anyway, as that would make packaging vastly more complicated. But for development, I am too old to work in an environment with at least some form of auto-completion, refactoring support (renaming at least) and go-to-definition.

Small setback: Turns out that for changes from git, the round trip to bzr and back produces a different git revision than you started with IF it is not done in just one instance of a bzr remote access. I suspect the same may be true for bzr->git->bzr round trips. So you do need to keep the metadata collected by the syncing process. It's much cleaner than for bzr-svn, though:
Revision ID information both for git and bzr is stored in marks files similar to those seen in fast-export. It seems that you can restore them with bzr/git fast-export, but I won't bet on that and include the git repositories used for syncing in backups. rsync to my real server, for starters.
User avatar
Z-Man
God & Project Admin
Posts: 11585
Joined: Sun Jan 23, 2005 6:01 pm
Location: Cologne
Contact:

Re: And now we have git.

Post by Z-Man »

Aha! I poked around a bit. The reason the git->bzr->git roundtrip did not work were the cherry pick merges I did in git. Git knows (at least) two dates for each revision, the Author Date (when it was written) and the Commit Date (when it was added to git). For normal commits, both are identical. If you cherry pick merge (or rebase, I assume) the Author Date is left unchanged from the source and the Commit Date is set to the current time. Bzr knows only committer and commit date, so only that makes it over. On the way back to git, that then gets cloned into both fields. Both make it into the sha1 hash, so that is now different.

Sooo... the way to avoid the problem would be to avoid two of the features that make git attractive.

It gets worse :( I poked around the source of git-remote-bzr and when they create a bzr revision, they use the usual random ID generation, and thus the git->bzr sync path is not reproducible. Two independent imports from bzr to git will produce the same git repository, but not the other way round. I already did some merges on git from 0.2.8 to master and indeed, on the bzr side, the merged commits have different IDs on 0.2.8 and 0.4 and merging on bzr will merge them again. With conflicts.

Apparently, you can pick the ID of a bzr commit freely, so it would be possible to base it purely on the git sha1 hash... But that creates more problems, because then mixed work in git and bzr may produce bzr branches that have identical tip revision IDs, but different histories, which can't be good.

I'll think of a procedure we can use so we can at least use git on master freely.

Does anyone have the new 0.4 bzr branches or git repositories cloned for work already? If not, I would like to completely clean the mess up by resetting the bzr branches a bit to the point before the merges from 0.2.8, redo the merges and completely restart the git repositories from a fresh bzr import. If you already have checkouts, no biggie, we'll just have a couple of duplicate revisions in the history.
User avatar
Z-Man
God & Project Admin
Posts: 11585
Joined: Sun Jan 23, 2005 6:01 pm
Location: Cologne
Contact:

Re: And now we have git.

Post by Z-Man »

OBVIOUS answer: Fix the problems in git-remote-bzr :) A kind soul already seems to have tackled the problem that "bzr->git->different bzr" gives different revision IDs, at least if you use the same git repository for import and export: https://github.com/utopia-repository/git-remote-bzr
haven't tested yet whether it works as I think it does, but if it does, it alone would have avoided the observed issues.

And it was rather trivial, apart from wrestling with Python without being able to see any error messages because the test framework eats them, to add the missing information on git->bzr export as revision properties and extract them again when re-importing to a different git repository:
https://github.com/zmanuel/git-remote-b ... 03dd3eb114
I'll submit that to upstream.
User avatar
Z-Man
God & Project Admin
Posts: 11585
Joined: Sun Jan 23, 2005 6:01 pm
Location: Cologne
Contact:

Re: And now we have git.

Post by Z-Man »

Done with the cleanup; the 0.2.8 bzr branches are unchanged, all others have had half a handful of revisions rewritten. If you do have them in a local copies, try to eliminate them by 'bzr pull --overwrite' resp. 'git pull --force', don't forget to rescue any local modifications first.
If you can't do any of that and the replaced revisions do make it back into the repositories, no worries. They'll just be that, a couple of useless redundant revisions. They should not make any more trouble.
The sync code has been updated as described, but there's one last catch: the git -> bzr -> git loop can only be as stable as git fast-export/git fast-import, and that one sadly isn't completely. So my own modifications are more a 'if everything else goes to hell, maybe we're lucky' safety net, made of sharks and tigers. The main thing is the new combined marks cache, that should allow us to work wherever we want and merge however we want.

Nevertheless, for the medium term future, in order to avoid problems:
- consider the 0.2.8 git branches read only, we're still on bzr for that one. The bzr branch is the one definite version, in case of trouble, bzr wins.
- merges from 0.2.8 to 0.4/master happen on the bzr side.
That's to give the sty hack branches the freedom to pick their own time to migrate to git.

I'll add those sty derived branches to the git repository too, it's best for consistency if the syncing/transfer happens all in one repository.

For master/0.4, do whatever. Of course, right now, I'm the only one with push rights on git. Hey, Light, want to be a maintainer over there? Just tell me who you are.
On the master/0.4 side, if anything goes wrong, the git branch wins. The bzr revisions from before the git branch's existence should be safe, though.

And yeah, right now, after I worked some more with git over the past weeks, coincidentally also at work, I also won't be sad if we leave bzr behind completely.

Automatic syncing between bzr and igt is not yet reactivated, I won't be at home for the next days and would not be able to pull the plug on the raspi should it go on a commit rampage.
User avatar
kyle
Reverse Outside Corner Grinder
Posts: 1876
Joined: Thu Jun 08, 2006 3:33 pm
Location: Indiana, USA, Earth, Milky Way Galaxy, Universe, Multiverse
Contact:

Re: And now we have git.

Post by kyle »

:) I've been very dormant the last few months, was happy to check in and see this, I will post more soon, and check IRC, something I've not done for a while either.
Image
User avatar
Z-Man
God & Project Admin
Posts: 11585
Joined: Sun Jan 23, 2005 6:01 pm
Location: Cologne
Contact:

Re: And now we have git.

Post by Z-Man »

Z-Man wrote:resp. 'git pull --force',
I mean 'git reset origin/master --hard' or 'git reset origin/legacy_0.2.8 --hard', depending on which branch you are on. Maybe do 'git fetch --all' before, though that should be automatic.
Post Reply