Ladle 55

A place for threads related to tournaments and the like, and things related too.

Moderator: Light

Post Reply
Concord
Reverse Outside Corner Grinder
Posts: 1661
Joined: Sun Oct 21, 2007 5:24 pm

Re: Ladle 55

Post by Concord »

k, time for a little math lesson.

e is just a number, it's approximate value is 2.718. It's defined as the value of a for which the graph of y=a^x has a slope of 1 when x=0. '^' means raised to the power of. So we are raising e to the power of -t/5. This means as t increases, as we get further away from the time the rating was made, the value of -t/5 gets smaller, and the value of e^(-t/5) gets smaller as well, and the rating gets smaller.

each rating is not +/- 1 but really +/-1*e^(-t/5), where t is the number of Ladles that have been played since rating.
So for Ladle 55 ratings, 0 Ladles have been played since the rating, so t=0 and e^(-0/5)=1, so a rating of 1=1.
Next month, t=1, so e^(-1/5)=0.8187307531, but we'll call it 0.82.

You could also do a decay of something like rating*x^t, where x<1 and t is the number of ladles since the rating, there's a lot of arbitrary values you could choose for x (the value of -5/e for x creates a graph equal to e^(-t/5)), or in my example, to replace 5 with, it's really just a matter of how fast you want ratings to decay. The decay I used has a relatively long memory, which I think is sensible for this use.

I attached a picture of the graph of the function y=e^(-t/5), where y is the value of the rating, and t, along the horizontal axis, is the number of Ladles since the rating.
Attachments
graph.jpg
Last edited by Concord on Sat Mar 10, 2012 7:44 pm, edited 1 time in total.
User avatar
sinewav
Graphic Artist
Posts: 6413
Joined: Wed Jan 23, 2008 3:37 am
Contact:

Re: Ladle 55

Post by sinewav »

Yeah, I think you should spend some time explaining why it matters. I'm not convinced we need to include time decay. Try and convince me.
User avatar
sinewav
Graphic Artist
Posts: 6413
Joined: Wed Jan 23, 2008 3:37 am
Contact:

Re: Ladle 55

Post by sinewav »

Raw Ladle Statistics.
Attachments
ladle_stats_55.xls
(20 KiB) Downloaded 97 times
Concord
Reverse Outside Corner Grinder
Posts: 1661
Joined: Sun Oct 21, 2007 5:24 pm

Re: Ladle 55

Post by Concord »

it's pretty obvious why it matters. the more recent experiences with the server are better indicators of how that server will perform the next time it is used.

It's just like if you look at restaurant reviews. If ten years ago, someone said the place was great, and yesterday someone went there and got food poisoning, I'm not going to weight those evaluations equally. Things change over time, the more time that passes between a review and the period we're concerned with, the more opportunity there is for the review to have become inaccurate, and it therefore should be less influential.
User avatar
Cronix
Match Winner
Posts: 467
Joined: Tue Jul 27, 2010 3:25 pm
Location: behind you!

Re: Ladle 55

Post by Cronix »

I would have understand it if you had explained it on german... Maths on english just sounds weird :D
Signature? wtf...
User avatar
sinewav
Graphic Artist
Posts: 6413
Joined: Wed Jan 23, 2008 3:37 am
Contact:

Re: Ladle 55

Post by sinewav »

Look, I phrased my last two posts wrong. I understand the formula, I know what it does. What I don't agree with is that we need the scores to decay at all. We are collecting purely subjective data here, and the person populating the brackets still has to make judgment calls. In addition, we need to keep things as simple as possible so anyone can step in and do this. We don't want people to think they are doing homework when they are just trying help out. (Besides, I don't see how your formula is really any better than Lackadaisical's "old_score * factor + new_score," which is much easier to manage and looks more attractive, IMO.) Personally, I would rather just sum them. It's easy enough for anyone to do it and the totals are still meaningful. What you are doing is massive overkill.

Already we have a problem where I decay isn't representing the server quality:

Code: Select all

Server     54 55   Score
ID VA1     -2     -1.64
ID VA2        -2  -2.00 
Both servers were used once, and both scored a -2. That's the real value of each server, -2. ID VA1 shouldn't be magically rated better because we didn't use it last month.

But whatever, you can take the time to decay the totals every month if you want. I'm already doing a lot of work for this event and I could use a hand.
Concord
Reverse Outside Corner Grinder
Posts: 1661
Joined: Sun Oct 21, 2007 5:24 pm

Re: Ladle 55

Post by Concord »

Concord wrote:it's pretty obvious why it matters. the more recent experiences with the server are better indicators of how that server will perform the next time it is used.

It's just like if you look at restaurant reviews. If ten years ago, someone said the place was great, and yesterday someone went there and got food poisoning, I'm not going to weight those evaluations equally. Things change over time, the more time that passes between a review and the period we're concerned with, the more opportunity there is for the review to have become inaccurate, and it therefore should be less influential.
btw: if you used lack's exactly as is, you'd have even more dramatic decay.

I calculated the decay, and I decided to use that method.

I also don't see what the issue with that example is. ID VA 1 has less of a chance of being bad than ID VA 2 next month, because it wasn't bad this month, which ID VA 2 was. There's nothing magic about that, it's common sense. If you want to know what my handwriting looks like, you don't look at something I wrote in first grade.
User avatar
sinewav
Graphic Artist
Posts: 6413
Joined: Wed Jan 23, 2008 3:37 am
Contact:

Re: Ladle 55

Post by sinewav »

Time versus actual use. If you are going to decay something, it's should be on it's own timeline. Not using a server shouldn't cause it's score to change. Think of your restaurant example. What if no one eats there between ratings?

I know lack's formula decays faster, I'm not an idiot. If we are going to use decay, use the easiest tool that's most meaningful.

And I'm questioning the value of decay versus ease of maintenance. Can you get that in your head? I guess it doesn't matter if you've volunteered to do it every month. Just don't be surprised if it gets abandoned on months you don't.
User avatar
ppotter
Match Winner
Posts: 451
Joined: Sun Sep 28, 2008 12:45 am

Re: Ladle 55

Post by ppotter »

Perhaps it's not necessary to have to use a formula to decay scores from older ladles, but in a similar vein discount scores from x ladles ago, say 3. Less work than using a formula, but keeps the ratings updated and relevant.

At the minute we have very little data in any case, with only one or two people rating each server, so with that in mind I should probably add my ratings for last ladle.

CT USA: 0 Not bad but there were several instances of lines disappearing and minor slides.
CT MBX: +1 No issues
CT LIV: 0 Didn't seem 100% stable, but wasn't bad.
User avatar
Lackadaisical
Shutout Match Winner
Posts: 823
Joined: Sun Dec 21, 2003 4:58 pm
Location: Amsterdam, Netherlands
Contact:

Re: Ladle 55

Post by Lackadaisical »

Concord wrote: btw: if you used lack's exactly as is, you'd have even more dramatic decay.
It just depends on the factor you use, it is actually exactly the same if you use a factor e^(-1/5) (so new score = this ladles rating + old score * e^(-1/5)). It's just a different way of writing down the same formula, with the advantage that calculating it takes a lot less work.
Concord
Reverse Outside Corner Grinder
Posts: 1661
Joined: Sun Oct 21, 2007 5:24 pm

Re: Ladle 55

Post by Concord »

You could also do a decay of something like rating*x^t, where x<1 and t is the number of ladles since the rating, there's a lot of arbitrary values you could choose for x (the value of -5/e for x creates a graph equal to e^(-t/5)), or in my example, to replace 5 with, it's really just a matter of how fast you want ratings to decay. The decay I used has a relatively long memory, which I think is sensible for this use.
btw: if you used lack's exactly as is, you'd have even more dramatic decay.
User avatar
INW
Reverse Outside Corner Grinder
Posts: 1950
Joined: Tue Jul 07, 2009 4:10 pm
Location: Charlotte, NC, USA

Re: Ladle 55

Post by INW »

So if a server has an amazing rating of +3 for 15 ladles and then receives a rating of -2 for one ladle. That -2 rating won't mean much because it has so many +3's?

But does that really make sense?

Let's say a server is really good for a long time but then turns to shit. Because its rating is so high from previous ladles, it is still deemed a "really good" server and its still used for the finals?

Or would having one "bad" rating doom the server because the rating is so recent?
PokeMaster
Match Winner
Posts: 638
Joined: Mon Jun 14, 2010 5:36 am

Re: Ladle 55

Post by PokeMaster »

the only difference between conc's formula and lacka's is the rate of the decay. Lacka's decays faster, so what that means is that the more recent votes will count more significantly than past votes, than compared to conc's decay rate. Conc argues for a slower decay rate because it holds longterm performance at a higher significance, but I'd say that performance in the most recent ladles is more important. If a server has a tracker record of great for 15 ladles, then shitty for the previous 2, I'd like for those 2 last ladles to have a significant factor in the overall score. (Yes concord, I understand that your e^(-t/5) formula stresses significance of the most recent input because it is a decay function, but lacka's does even more.)

Anyway, the rate of decay is a bit of a frivolous argument because it's impossible to say what exact decay will give us the best results. And in any case, server stability over time realistically can't be accurately predicted with any function, so it's best to choose a formula and get on with it, and preferably choose one that is easy enough for anybody to use (as sine said).
Image Image Image Image Image Image Image Image
Image Image Image Image Image Image Image Image Image
Concord
Reverse Outside Corner Grinder
Posts: 1661
Joined: Sun Oct 21, 2007 5:24 pm

Re: Ladle 55

Post by Concord »

it's really not any harder.

anyway, having only two raters for a server makes any score almost meaningless. For this to mean anything, we need a lot more ratings
User avatar
INW
Reverse Outside Corner Grinder
Posts: 1950
Joined: Tue Jul 07, 2009 4:10 pm
Location: Charlotte, NC, USA

Re: Ladle 55

Post by INW »

Concord wrote:anyway, having only two raters for a server makes any score almost meaningless. For this to mean anything, we need a lot more ratings
ya, 3 ratings out of more than 100 players.
Post Reply