Let's make the official resource repository useful again

[epsy]

Judging by how everyone and their mother is putting up their own resource repository rather than using the official repository, it is way overdue to ask ourselves why is no one uploading their map files etc to the official repository. Please help me in answering this question, so that we can improve the repository and leave no doubt that if you want to upload something for the game, it is the place to go to.

So, I will start:

Uploading a resource is inconvenient

If you ask around, practically no one knows how to get started uploading to the repository. The instructions are away from where you would expect(ie. somewhere on http://resource.armagetronad.net/), on this wiki page. The instructions there send you to a page asking you for a username/password, and in order to get one, you have to PM Luke-Jr somehow. Contacting someone to create an account creates a large delay between you wishing to upload something and you actually being able to carry that action out.

My solution to this is to not use registration at all and instead use the elaborate authentication scheme we have already made up to identify uploaders. This means resource authors would be GIDs and that you would not be able to upload any more resources with non-GID author names. (For anyone interested, I will not use a system in which the user has to give their password to anything but their own game client.)

You can only upload maps

Since the resource repository got started, cockpits, config files and textures were added to the resource system that the game handles. You can add none of those through the current web interface.

For cockpit files and config files this doesn't require much thinking (or does it?). For textures however, we probably have to implement a rating system(eg. what age groups would we rather not have see this) at this point, let the resource repository advertise it somehow, and have the client trust ratings from a configurable list of repositories(the official one being there by default).

Browsability is poor

This isn't too important, but it is nice to be able to browse the repository in a web browser. Map and cockpit previews as well as highlighted, documented, and interlinked configs along with usage information for each resource would be neat.

I can't think of any more right now. If you can think of anything please reply with it. (You don't have to structure your post like mine.)

[Light]

A bit of what you stated and a tad more.

I'll start with something you've said, and that's different file types being unsupported. I host tournament and other server configs on my resource, which provides players with an easy way to set up servers, keep an auth list, etc. Also being the same location as the maps is just convenient, even though I personally don't use them together, though I know you can.

It being on a remote server from my game servers is a little bit of a problem for me. Servers such as my labyrinth will add maps to the resource so users can download them. Uploading constantly to a remote server would just be pointless when it can store them locally. As far as bandwidth goes it wouldn't be bad, but you would have to be able to run with that delay between map changes, and I wouldn't like that very much either.

Finally, lets get to my main reason. This is a bit out of order, but whatever. My web server being on the same server as my game servers (serverserverserver) is very convenient. I know it's not often, but there is sometimes downtime on the main resource, which can cause users to be unable to play or the server to time out when attempting to check for files. When my game servers are up, so is the resource, so I have nothing to worry about there.

As for browsing through on browsers, I don't see much of a problem with using the default listing as they do. It's not all pretty looking, but it's all that's really necessary. Creating thumbnails and everything just seems a bit pointless to me.

If you did go that route though, it would be nice to have uploading and everything set into the same manager. Until this post, I never actually knew where to upload to it. Hell, the only reason I knew where it was is from hosting servers. Building a manager for this wouldn't be hard by any means, but authentication may be an issue.

People are against using GID's, but they like to store things by them, which goes against their need to not log in. I know it would have to be someone trust worthy enough, but Idk .. You could go and set up accounts users can register to upload to, but then everyone would be stealing each others names and it would probably be the main target of trolls.

Which brings me to my final reason of liking to use my own. Sometimes users share the same name in-game. I wouldn't want another user uploading as me or the other way around (just saying whoever was first), and if you both were uploading different versions of the same thing or just sharing the same name for some reason, it could be a pain. I like that people can trust settings making me or anyone else admin and shit wont be stored in the maps.

For the regular user, I guess it's inconvenient to have your own resource server, but for those who already have web servers and whatnot, I would argue that it's more convenient. I've never actually used the main repository because I didn't know how to upload to it, and I didn't feel like having to bother with sending people maps to have them add it or whatever. I wasn't sure how they did it because I didn't see any info on it, so I just went my own route. Maybe it makes me biased a bit, but I wouldn't bother to go back now.

And a little touch on the settings thing. I've uploaded some of other people's maps that made them admin in their map settings, after stripping it out. I probably couldn't do that there, and shouldn't if I could because overwriting would be a dick move, but it's nice to be able to avoid that. For example, using Cody's sumo map.

I think I'm just rambling random thoughts now, so I'm gonna shut up. (:

Edit: I lied .. I wanna throw one more thing in. I'm pretty sure Lover's resource would be closer to what you're wanting. He has a preview on there and everything, though I'm not sure what all he allows. I never used it, but I've caught a glimpse of it in the past, and I think it's worth a mention here.

http://vertrex.tk/aa/resource/

[sinewav]

Back when I first started making maps I really enjoyed using the repository. However, it soon ran into problems because the uploader had a size limit. All the Adventures of TRON Racing maps exceeded that limit and I had to bother Luke-Jr for a manual upload every time. Sometimes it was a long wait and I always felt like a nuisance. It wasn't too long after that the site went down and never seemed to get back up with any regularity. The browser and uploader broke, everyone lost interest in maintaining it, then Tank and Luke-Jr disappeared.

I really don't like hosting my own maps because I would rather them be in some sort of community archive. I probably put in more hours than most when it comes to map design and I think it's good to have them readily available and not dependent on whether my site is up. Besides, browsing the resource repository is inspirational and stimulates creativity!

One thing though, the repository can't have the same issues with downtime and broken utilities, otherwise why bother? There are lots of threads here complaining about just that. I think thumbnails are important.

Here is an incomplete breakdown of how we got here:

• philippeqc suggests improvements, comments that beta.arma is down (happens a lot). wrtlprnft suggests armathentication, as epsy mentioned above.
  
  Luke-Jr makes a public announcement about accounts, some comedy happens.
  
  Luke's server dies along with the repository. Tank offers to host at home, then disappears. Nothing has been working right since (July 2009).
  
  LOVER$BOY makes a public repository, because over a year has gone by since anyone thought about fixing the problem. Z-Man approves.
  
  compguygene, late to the party as usual, decides to make a repo, or a clone, mirror, or who knows what actually happened -- but good ideas where exchanged between gene, Lover$boy, delinquent, and Z-Man (Important Read).

I would like to see a public repo, with proper wiki instruction, and easily visible on the main site.

[LOVER$BOY]

@sinewav and @light
Just getting mentioned makes my day for now. Really happy to hear my work is actually noticed by someone else, other than Z-Man himself

@light: Yes! I thought trolls would violate my rules of hampering into someone else's author/category/ directory. So instead I currently made it so only REGISTERED users of http://forums.vertrex.tk are the only ones capable of making maps and uploading to our repository. That just seems fair since you said it yourself, trolls will be trolls

@sinewav: Ha... I had no idea the top bits of what you said but I do now! Wow... now I feel proud to be part of the community!

Sine about your last comment:

I would like to see a public repo, with proper wiki instruction, and easily visible on the main site.

Could you please elaborate since I am slightly unsure of your real intentions in what you said. I don't want to assume since my imagination is too strong... got me into so much trouble I gave up trying to assume things from then on.

[sinewav]

I would like to see a public repo, with proper wiki instruction, and easily visible on the main site.

Could you please elaborate since I am slightly unsure of your real intentions in what you said.

I don't have intentions, I just have this opinion: We should have a repository that addresses all the things mentioned so far in this thread and others. If I were to imagine an ideal repo, it might look like this:

• It has a central location and is of an official nature, like resource.armagetronad.net or something that let's people know "this is the main place for resources."
• It can be maintained by a team of people, even if that team is only a duo.
• It is mirrored.
• It uses armthentication.
• It has documentation, server-side and client-side. Users should have little difficulty logging in, uploading resources, and finding them. If something breaks, knowledgeable people should have no trouble finding the problem and fixing it.

And ideally it has the following:

• A browser that also outputs a code snippet for settings_custom.cfg or similar.
• A map preview, because why not? I think it helps when there are multiple versions of a map.
• Other cool stuff.

LOVER$BOY I have not looked at your repository, but I hear it is very nice. You should definitely be on the team of people who help rebuild the official repo.

[LOVER$BOY]

Ah yup! I see what you mean and they make sense perfectly! Good ideas here!

Thanks for the compliment. Yes, I hope to help out in the main line some day if ever I'm needed

Now here's something that I've PMed Z-Man although he isn't satisfied with my chain of thinking:

I was thinking of using a script drive which works like what comp discussed a few months ago in the forums. Only this time I was thinking of using a script (php) that does the work.

What I mean is write up a script (which I can do easily) and have the list of all the currently used resource repositories for the servers. Then when that script is contacted, the script will parse the given MAP_FILE and search through it's local first to see if that map exists. If it doesn't then it will use the list of other resources and contacts them instead. If they contain that map, then it will load that map and output it so the server will start downloading it.

This is also an easy way to send error reporting codes like 404, 403, 502, 504, 512, etc. So that the server/client don't have to download some false or broken maps since validation will be taking place when the fetch.php script is called.

Say for example this:

Code: Select all

http://vertrex.tk/aa/resource/fetch.php

When the server is looking for a map it will contact that script as so:

Code: Select all

http://vertrex.tk/aa/resource/fetch.php/{map_file}

Then the script will parse and check if {map_file} exists. If not then it will contact the other online resources:

Code: Select all

crazy-tronners.com/resource
roadrunnerz.net/resource
tr2norigins.net/resource
adventuresoftron.com/aa/resource
tronner.com/data/repo
devel.game.osvaldojr.com/tronsvr/rse
shenanaboosh.me/armagetronad/resource

The list so far only contains what I know but needs updating

[LOVER$BOY]

This was Z-Man's reply and what I replied with:

UPDATE

Thanks, I'll look into it when there is time.

IDEA

Dunno. What would be the benefit? All I can see it doing (in the form presented) is replacing multiple points of individual failure (the individual repositories) with a single, central one (your script/cache/storage). And adding a bit of latency.
Now, as a fallback mechanism, it could be a thing: first, check the original URI and only if it does not exist, check http://www.resourcerescure.net/pleasefi ... =<original uri>, that would be easy enough to code in. But then, how do you fill the rescue repository? Clients and servers would need to report actually used resources to the rescue repository, which is a bit iffy from a privacy standpoint. What I'd rather like to see instead is decentralized resource caches: instead of going to some URI, clients would get the resources from the game server itself.

Ah... I see what you mean... That is a bit of a snag...

On the other hand your next idea...

Do you mean something like: http://vertrex.tk/aa/resource/rescue.ph ... .aamap.xml

Good idea. This way is much easier than trying to gather all the available resources. Silly me

But why use url? I actually thought of another plan. Use rescue.php?resource=...&file=... That way when manually parsing it can easily. Look at the sources so you know what I mean. Although it only supports fetching .aamap.xml maps but I can modify it to also process .cfg files as well...

Sources
rescue.php
fetch.php

Anyway I've got both that fetch.php and rescue.php working. The one thing I want to fix is that DTD which is much simpler though also very easy to hack with. Check them out here (by the way, I don't have these maps on my server):

Rescue:
Resource: resource.armagetronad.net/resource/
File: Anonymous/polygon/regular/Weird-1.0.1.aamap.xml

Fetch: Anonymous/polygon/regular/Weird-1.0.1.aamap.xml

UPDAT
I've tested it out with my local server and it works!

The only problem is is has time lag and then jump around like crazy but settles down again once map is downloaded successfully.

[LOVER$BOY]

Now I've modified that script and decided it will not be useful. Just think about it the time it will take going through each of those resources... man, the server will skip map download and the client might return error of map download failed. Instead, I now am fully inclined to believe rescue script will work the best for those who couldn't do like compguygene suggested in his repository post (sinewav posted it far above).

Right now it only loads the map files in webpage but doesn't actually store them. Whether you want the retrieved map to display or not is totally up to you. My original intention was to store maps directly so to save script time but then I realised a small flaw in my plan. What if the map rescued was a fake and the real map or author is someone else? What if that map is a stupid nutcase while the real one is something beautiful? So I dropped that silly idea of mine and instead rely on merely displaying the map code if it is found or 404 not found error.

You can test it out here: http://vertrex.tk/aa/resource/rescue.ph ... .aamap.xml

That map does not exist in my local repository. You can quickly go into browser and check it out if you doubt me. It's easier to prove than with words alone

Anyway I'll leave things here and wait for replies from everyone else

[Word]

If you can think of anything please reply with it.

If you're going to include textures, why not .ASE- and .WAV-files as well? Obviously those would be likely to need some rating system too.

[sinewav]

If you're going to include textures, why not .ASE- and .WAV-files as well? Obviously those would be likely to need some rating system too.

I like where you are going with that. Maybe create place for moviepacks/moviesounds so we can integrate that in future versions of ArmagetronAd? Seems like we are getting ahead of ourselves with, but it highlights the need for an extensible framework.

[kyle]

DTD's

This is the main problem that caused ct's repository to diverge from the official one. Sure we did have our own prior to the sty hack, but a lot of our maps become broken because the location of the pigsty DTD differed. Originally i believe groundpig stored it in AATeam, but it was since moved to some other location. So none of our newer maps at the time could be uploaded.

Sometime I did manage to scrip the uploading process using curl, which made it easier to upload maps. After I made that script my new problem became my DTD's I made for sty+ct.


backwards completable

The new system should still allow the current authors to add maps, upon request. So I should still be able to use ct and kyle as my authors likewise sinewav should have access to turtle Yes this means someone probably needs to manage it, but it should be minimal work.

Again i do have armaauth integrated with the current system here. and a small database stores which folders users have access to.

my maps
Now we also should be able to syn the repositories somehow. The old way to sync the master into your using rsync works. With all the ct / ed / kyle maps not in official The only way I would use a new system is if the current ones where synced.

Easy upload
It must be easy to upload maps or multiple maps. so hopefully it will allow for a curl script in a directory with maps?
Also armaauth for new people to add resources wiki pages should be updated to use GID for the author.

[Light]

Lover, there's an alternative to making it take a long time. This could be a bad thing for large map files, but you could just query all of the resources and see if any contain it.

I made an example script of this idea:
http://localhost/resources/?item=/Animu ... .aamap.xml

That example will pull it from your resource actually. But you can see the speed is decent since you don't wait for a response before checking the next. There's still a slight delay, but that would probably also be better from a business connection rather than my home.

I used the resources listed here:

Code: Select all

	$resources = array(
		"resource.armagetronad.net/resource",
		"crazy-tronners.com/resource",
		"roadrunnerz.net/resource",
		"tr2norigins.net/resource",
		"adventuresoftron.com/aa/resource",
		"tronner.com/data/repo",
		"devel.game.osvaldojr.com/tronsvr/rse",
		"shenanaboosh.me/armagetronad/resource",
		"lightron.tk/resource",
		"vertrex.tk/aa/resource"
	);

[sinewav]

I don't have the experience in this area to be much help, but I volunteer for whatever I can do. I can write user docs if the tools are explained to me correctly. I can manage something here or there if needed. I'm generally too busy to actually play the game (except for a round or two of Sumo in the middle of the night), but I can usually find a few minutes every day to work on something Arma related.

Does someone want to make a roadmap (blueprint?) outlining the steps we need to take to make this happen?

[Lucifer]

Here's my two cents.

Since Loverboy apparently has the most functional repository, would you be willing to donate all code used to start an official repository? You'd have to make it GPL (if it's not already)...

Then we take it into the core project stuff, make Loverboy a developer with write access to the repository code, and put him in charge of running the main repository. When that happens, people can submit patches and stuff and Loverboy can review and merge as needed.

I'm sure I can talk Neal into running a mirror, depending on bandwidth usage, of course. Hell, he'll probably volunteer. Neal's server is quite reliable. I know because I do a lot of the admin stuff on it.

[LOVER$BOY]

@kyle: I saw that it's pretty epic!

@Light: fetch.php does exactly that. Truth be told Light the script actually doesn't take a long if used online but it does do it slowly from home. Which is why I'm against home hosted and look for something business class or something higher. I am restricted from a lot of uses from PHP since my hoster is a bit of a freak with security. For my host, 8ing, is a business class and has fast paced internet and high speed download/upload which is why I use it sometimes. However, if I do too many connections the security script will catch it, send it to him and he will ask me "What were you doing?".

@Lucifer: Hello Lucifer, fancy seeing you here

Sure but I do not have a GPL Free License for it yet. It is like this:

Code: Select all

    <one line to give the program's name and a brief idea of what it does.>
    Copyright (C) <year>  <name of author>

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with this program.  If not, see <http://www.gnu.org/licenses/>.

If it is, can you tell me how I would apply it to my codes since I don't think it's a simple "COPY" and "PASTE".