Random To Do List (RTDL) Suggestions Thread


#21

Missed this question earlier.

With your pool of (say) 7,000 achievements; these are sorted by ratio from high to low; then achievements are selected from a “percentile” slice of those achievements.

The top tier as currently defined would select one achievement from your 350 highest ratio achievements.

The updated proposal would select from your top 700 highest ratio achievements instead.

The end result of this is that the highest ratio achievement has a larger range to select from. This means that it can end up with a lower ratio than it currently does, and means that it will be less likely to throw the entire list out of balance, total score wise.


#22

In regards to normalizing the maximum score.

You mentioned in an update in the other thread that you
"Attempted to normalise the potential score for every list to between 5500 and 6500, but this is not always possible."

Can you describe what is being done in more detail?

I like this idea.
Keeping maximum scores close could be an issue in smaller length contests.


#23

Certainly.

  • I query TA’s API for a list of achievements with the set criteria, and get the results back.
  • I then tally the ratios of all of the results.
  • If the results are between 55.0 and 65.0 then that’s your list.
  • If the results are above or below, I cache those results and try again.
    • If I fail to find a valid set of achievement after 5 tries, the ratio threshold (at whichever end the lists are bumping against) is bumped by 1.0, i.e. if the lists I’m getting back are below 55.0 then the bottom threshold is lowered to 54.0, and vice versa for the upper threshold.
    • The cached results are checked against the new thresholds to see if I’ve already found a valid list (saves some hits on the API and also forces the use of “close but not quite” results I may have lucked into earlier).
    • If not, start querying again, and we continue to loop this way until we get a valid list.

TL;DR if I can’t get lists within 5500/6500 value then the thresholds will keep getting bumped until you get a valid list. On other words, that threshold is a soft limit.

How so? I’d have thought a single blowout list in one month for one person would be more detrimental in a shorter contest, it would basically guarantee the win for that person.


#24

Yes. I think we are saying the same thing. You just said it more clearly than I did :smile:


#25

Love the RTDL, whatever you decide on will be great I’m sure!
A couple of thoughts -

  • More achievements per month would be good, how about 50? I find that on months when I don’t have a lot of time I grab the easy ones in the first week or so then do nothing with the rest of the month. Having more (in the same ratio proportions) would give more to work on.
  • To go with the first idea, and offset the larger number of lower ratio achievements, how about some sort of weighting system beyond 100*ratio ? An achievement like A New York Minute is worth 564 points in the current competition, but is much, much harder to get than 4 x 125 point achievements. Perhaps scoring based on the square of the ratio or something like that would make the harder achievements more attractive to go for!
  • Shorter competitions sound good, but maybe three months is too short to offset a poor month, I feel that 6 would be a better fit and allow more movement in the rankings over the length of the competition.

#26

I could see something like 4 month seasons with 50 achievements each, weighted so the majority of them are in the lower end of your achievements. Doing the scoring based on the square of the ratio would certainly mix things up too, and you’d have to decide whether to just go for those harder ones, or work on lots of little easy ones.

I like it. With shorter seasons we can also mix things up and change things around on a more frequent basis - so we can experiment with exotic scoring and huge lists.


#27

I’d be down for 4 months - Sept through December for 1st Trimester.

I’d have to see the points play out in some real life examples before I can weigh in. Conceptually, I like the idea - I’m just having a hard time visualizing how that looks in practice.

I’m already stressed out about 25 achievements that I can’t get - at least with 50 - I’d feel like I could make more progress.


#28

I like the sound of 50 each month. My personal issue with the current format is the new 500 achievement limit! I personally love the RTDL and the fact it takes from the majority of my games. I have taken a handful of games off that I no longer have or really don’t want to play. I have been challenging myself to stay in the top 10 overall, which has been a fun and difficult battle. With this sudden trend of people dropping their pool massively those people are top of the monthly board and over time would obviously become top of the overall board. I know I could do the same, but I don’t want to! It feels that those of us that want to keep it to a larger pool of achievements from most of our games now have no chance of being at the top end of the tables. I thought the idea was a random to do list not a very controlled idea of random! I love the fact it gets me back to games I haven’t played in years. I understand this rule was put in to allow those with smaller games collections to enter. So I do have what I believe to be a fairer way about this. Everyone is only allowed to remove a certain percentage of their overall achievement pool. I think around 10% would possibly be a good area. Would like to know if others feel the same as myself, I know the other UK guys I speak to do!


#29

My biggest issue is that when I have 400-500 games, that leaves a lot open to grab. Many of those I’m not as excited to play as other titles I own. And I wouldn’t want to play a sequel to a game I never played the first entry in.

It’s a tough balance.

You could find some way to have a conversion factor based on pool or game library size.

Maybe the median or average for the draw could establish the 1x - then ratio the deviation from that to get more or less points based on being above or below that value.


#30

Well put, Keith. I suspect there are several people feeling this way.
For those that want to compete it’s starting to become a contest of how to manipulate your game collection. That could be fine, but it’s not the original intent of the contest.

I don’t think a bonus or penalty based on pool size is the answer. That can also be manipulated.
In fact, why not make everyone’s possible points the same? You have 7000 (or whatever) possible points to earn each month. Basically just get a ratio between 7000 and your max score and multiply that times the points you would normally get for each achievement.

I think an increase in minimum pool size is also in order. I would try something between 1500-2500 achievements.

There’s still room to experiment, of course. Especially with shorter contests.


#31

Firstly: the 500 achievement limit is not new - the rule was there from the start, but only enforced and visible later in the competition. People were already “abusing” that particular statistic, I just made it more visible.

Let’s play with some stats and see how the “randomness” breaks down.

A pool minimum of 500, and the breakdown of the draws is:

{count: 12, min: 50, max: 100},
{count: 9,  min: 25, max: 50},
{count: 3,  min: 5, max: 25},
{count: 1,  min: 0,  max: 5}

If your pool is exactly 500, then:

  • 250 achievements make up the 50-100 percentile range.
  • 125 achievements make up the 25-50 percentile range.
  • 100 achievements make up the 5-25 percentile range.
  • 25 achievements make up the 0-5 percentile range.

The draws are as follows:

  • 12 achievements will be drawn from a pool of 250 (4.8%).
  • 9 achievements will be drawn from a pool of 125 (7.2%).
  • 3 achievements will be drawn from a pool of 100 (3%).
  • 1 achievement will be drawn from a pool of 25 (4%).

Here’s the changes I’m considering right now:

A pool minimum of 1000, and the breakdown of the draws is:

{count: 12, min: 50, max: 100},
{count: 9,  min: 25, max: 50},
{count: 3,  min: 10, max: 25},
{count: 1,  min: 0,  max: 10}

If your pool is exactly 1000, then:

  • 500 achievements make up the 50-100 percentile range.
  • 250 achievements make up the 25-50 percentile range.
  • 150 achievements make up the 10-25 percentile range.
  • 100 achievements make up the 0-10 percentile range.

The draws are as follows:

  • 12 achievements will be drawn from a pool of 500 (2.4%).
  • 9 achievements will be drawn from a pool of 250 (3.6%).
  • 3 achievements will be drawn from a pool of 150 (2%).
  • 1 achievement will be drawn from a pool of 100 (1%).

Based on the percentages it will be much harder to predict your draw with 1000 in your pool and the adjusted brackets, particularly in the top bracket.


It looks like the middle bracket (the 9 achievements bracket) is actually a bit too narrow, based on these figures, so a further change might be:

A pool minimum of 1000, and the breakdown of the draws is:

{count: 14, min: 50, max: 100},
{count: 7,  min: 25, max: 50},
{count: 3,  min: 10, max: 25},
{count: 1,  min: 0,  max: 10}

If your pool is exactly 1000, then:

  • 500 achievements make up the 50-100 percentile range.
  • 250 achievements make up the 25-50 percentile range.
  • 150 achievements make up the 10-25 percentile range.
  • 100 achievements make up the 0-10 percentile range.

The draws are as follows:

  • 14 achievements will be drawn from a pool of 500 (2.8%).
  • 7 achievements will be drawn from a pool of 250 (2.8%).
  • 3 achievements will be drawn from a pool of 150 (2%).
  • 1 achievement will be drawn from a pool of 100 (1%).

I think that looks a bit better. Thoughts?


On the maximum pool value - I’d floated 7000 as the maximum based on what the current pool values are (basic rule of thumb was they value ending up between 5500-6500). Given the pool size and distribution changes though I’d need to do some test scans and see what a reasonable new maximum should be. I think it will be significantly lower than 7000, probably closer to the 6000 mark with the average pool coming in at 4500-5500.


#32

To address a couple of the other points:

Making a list for everyone that had the same value would be actually pretty difficult with the API I have. It’s simply not set up for that kind of query. I’d need to keep polling it until I got a list that was close, and given the variance we already have it might not even be possible to draw a list for some people to match a specific value without adjusting the brackets on a per person basis. That’s getting overcomplicated in my opinion.

I have no way to enforce a restriction such as “only remove 10% of your collection” and have no desire to do this. I feel that the pool size and the brackets are the most important values we have to play with here and adjusting them should alleviate most concerns about gaming the system.

No matter what we do with the restrictions and rules people are going to push the limits of them. There’s no real way to avoid people pushing the boundaries beyond setting them to a reasonable position to limit the the influence of this type of play, but this also needs to be kept in balance with making the competition accessible to the most people.

There will be a significant number of people who legitimately have a pool around the 1000 mark due to a high completion percentage, and I’d hate to force them out of the contest. A pool of 1500 or higher is too high I think.


#33

I think you were addressing my idea here, and I didn’t make myself clear. My suggestion is to normalize the points after the lists are drawn. So everyone gets their list and their max possible points may be anything from 5500-7000. If player A is at, for example, 7000 and let’s say we want everyone to have a max possible score of 6000. Multiply all this player’s points by 6/7 thus changing their maximum to 6000. In this way, everyone could have a max score of the same thing, 6000.


#34

I see what you mean. I suppose we could do something like that - pick an arbitrary total value, divide it by the total ratio of the achievements selected, then multiply it back again for the adjusted achievement value.

My concern with doing that though is that it makes what is currently a fairly simple calculation quite a bit more complex, and thus more difficult to understand for most people.

I actually proposed something along these lines a while back as an alternative way to calculate the TrueAchievement score for achievements, and my difficulty there was explaining exactly how it worked too.


#35

Yup. And you’re right. It might be confusing to people. You would have to come up with a good way to present it. But it might be worth it to get everyone the same maximum score.
A more radical change would be to do away with points altogether and just base the leaderboard on the number of achievements from your list you’ve popped. Just gonna throw that one out there too :slight_smile:


#36

Looks good. With a minimum 1000 pool (and a max possible score) I think this would be an improvement if you don’t go for a more radical change.


#37

Still feel good about 25 achievements per month?

Also - three 4 month runs?


#38

OOF controversial!

I don’t think that will fly for most folks - I could set up a seperate leaderboard to track that way though.


#39

Given the figures above, if we change the number we draw we’ll also need to adjust the minimum pool size to compensate. If we upped the draw to 50 achievements it would cause the same issues as reducing (keeping) the pool minimum to 500, in that the chance to get specific achievements you want is greater.

I’m liking this if for no other reason than it fits neatly into this year after the current run finishes. :wink:


#40

Yeah, I think I want to take that suggestion back lol. Getting extra credit for the harder achievements is part of the fun.