The discussion has moved over to Discord

Home Forums The Archive Overload Early Release Version Comments and Feedback Point-based scoring system for challenge mode

RSS Feed
Viewing 12 posts - 1 through 12 (of 12 total)
  • Author
    Posts
  • #15723
    YinutYinut
    Participant
    Topics: 10
    Replies: 55

    Several people in the Overload community believe that a point-based scoring system would be better for challenge mode overall for a number of reasons.

    Firstly it would remove the need to significantly alter the robots in order to balance them.

    Secondly it would remove the need to “fish” for good robot setups. Currently robots such as Reavers and Guardians are incredibly lethal compared to goblins or Harpy-Vs, which means that if you encounter Reavers or Guardians early, you might as well restart the run. Now if Reavers or Guardians were proportionally worth more points than goblins, then there wouldn’t be a need to necessarily restart the run.

    Thirdly it would fix luck-based high scores since weaker robot setups would also be worth significantly less points. It would also have the added benefit of having some interesting high scores, for example a run with 69 kills that’s filled with heavy-hitting robots could be worth as much points as a run with 300+ kills with weak robots. NOTE that the important stat would be the score and while kills could be showed in the leaderboards, the stat that determines the position in the leaderboard is the score.
    For example:
    #1 Mike 23784 (87 kills)
    #2 Zerglord69 19340 (249 kills)

    The amount of points that a robot is worth could be fairly easily determined by analyzing the data that several players have been collecting lately. The important data point would be the “average damage per robot”.

    In summary I believe a point-based system would greatly benefit challenge mode by increasing consistency and reducing certain frustrating elements that the mode currently has.

    #15724
    maniak1349maniak1349
    Participant
    Topics: 10
    Replies: 18

    tl/dr: Point-based system is good thing on itself but accurate one will take too much work to make. Leave scoring as it is – run count will negate RNG. Add simple unprocessed stat records to highscores to let people see what is that score made of.

    I do agree in general with the fact that point-based system will give a better representation of player’s performance during a CM run.

    But the thing is – it will require a huge effort to implement accurate point-based system that will take into account all important aspects of the run. And those are quite numerous: robot strength (which should be somehow unbiasedly determined first), total strength of the robots spawned during run, amount and type of items and power-ups spawned, spawn timings etc. And there also are things that cannot be properly taken into account. For example – weapon choice. Player A choose to use Weapon X during the whole run and Player B chose Weapon Y/Weapon Z combo. Nothing prevented them from choosing otherwise other then their free will. Weapons are not equal in performance as they are and they may be more or less suited for the map in question so they will affect performance greatly. But its really hard to express this things in actual numbers.

    Completely accurate point-based system is a huge work. System that will take into account only some aspects of the runs will leave at least someone dissatisfied. So my vote goes for the simplicity – just leave things as they are, and allow the run count negate the random factor. If someone wants those pretty number on the leaderboards – they will get it with no regard to the cost – let them sweat. For me fun from the runs themselves much more important.

    And for the issue that was mentioned in chat – to let people see and measure what kind of highscore they are dealing with – luck based or hard earned – I suggest a simple solution : just store those Ctrl+C/Ctrl+A stats alongside with the highscores – without any kind of processing and presentation – just in plain text as they are. I’m sure someone will make a tool to inspect and measure those just in no time. And later if there will be free time and there still will be high demand for it stat processing may be integrated into leaderboards.

    PS: I’m still hoping for more stuff in stat records that I have mentioned before, but I do realize that there are much more important things to do – like finishing and releasing the game.

    #15725
    rapturraptur
    Kickstarter Backer
    Topics: 29
    Replies: 134

    I don’t think adding a point-based scoring system is a good idea.

    • It’s impossible to come up with appropriate point values for bots; bots with reflex are probably more dangerous in confined maps like caverns than in open maps like hive, for example, but it would be weird to have different point values for different maps (and unsustainable for community maps). Any system of point values would still make some bot combos more desirable than others for a particular map, difficulty, play style, …
    • There’d still be an incentive to fish for: getting invuln right before an ambush or triple superbot; having more goblins spawn right behind you than tritons; getting a cloak and homing bots in the same map; …

    Ultimately, I think the desire for a point-based scoring system comes from a desire for the challenge mode leaderboards to be competitive in a rigorous way, but I think this kind of competitiveness is the wrong criterion to optimize for. As long as they list the max score rather than average, pilots who play more will have an advantage over and above their actual skill. The leaderboards are a fun way to provide a goal across runs, but they’re not a serious comparison of pilot skill.

    I think a better criterion is to make each run as fun as possible. While that includes keeping the difficulty relatively consistent (which they have been doing by tweaking the spawn system), it doesn’t include awarding more points as a consolation prize: I’d rather not get two reaver ambushes in the first 100 kills at all than die on the second one and see that I got a lot of points for the first one!

    #15726
    LemurFromTheIdLemurFromTheId
    Kickstarter Backer
    Topics: 3
    Replies: 24

    WARNING: prepare for lots of text.

    TL;DR version: score >>> number of kills.

    *** A point about leaderboards ***

    Leaderboards, by their very nature, are ordered lists, and they need one stat the order is based on. Even if you have one primary stat and one or more tiebraker stats, they can be mathematically reduced to one statistic.

    In this post I will refer to this stat as X, regardless of what X represents.

    To be clear, leaderboards can be based on just about any stat, and they still function perfectly fine as leaderboards. If X is number of kills, player A has more kills than player B, then it’s perfectly fine to say that A is “better” than B, regardless of how they achieved that: it’s the kills that count, not the time you took to get those kills, the number of tritons you destroyed or the weapons you used. A might be better than B simply for understanding what the actual goal is.

    The important thing to realize is that X affects gameplay. It is the main objective of a CM run. You might be a player who doesn’t pay any attention to it, or you might occasionally play with specific personal goals, but the playerbase in general will always try to play in a way that maximizes X – and that has repercussions. For example, if X is the number of kills, a lot of players will give up on a CM run if they face a difficult spawn that makes getting lots of kills very difficult.

    It’s also important to realize that trying to encourage a specifc playstyle by choosing X that rewards it is the exact same thing as trying to discourage every other playstyle – and that’s a bad idea. For example, a reward for killing bots particularly quickly might sound like a nice idea – but isn’t a slow, methodological and careful approach just as valid a way to play the game, and why should that be penalized?

    X should be as playstyle-neutral as possible. It should reflect the challenge the player has overcome as accurately as possible – and nothing else.

    If X is not one of the trivial stats like number of kills, I would call it score.

    *** Score and points ***

    I would implement score by giving a set amount of points for each robot destroyed, depending on type of robot. For example, a Goblin might reward 100 points, a Gorgon 300 points and a Guardian 1250 points.

    That’s it.

    *** Reasons to use score instead of kills ***

    1. Much more accurate representation of performance

    I recently made a program to track my CM stats (I know others use spreadsheets, but I hate spreadsheets), and based on a somewhat limited data, Guardians deal about 15 times the damage per robot than Goblins. Why, then, do I only get one “point” for each? The simple fact is that kill count is an extremely inaccurate representation of general player skill. The correlation is there, but the variance is incredibly high. If one of my runs ends with 100 kills and another with 200 kills, which one actually represents better player skill? You simply can’t tell from that number alone.

    On the other hand, score can represent the actual challenge level of each bot quite accurately. AFAIK, several people have gathered stats for a long time (frankly, Revival should be gathering that data from players too), and that would provide an excellent starting point for a score system.

    Obviously no score system can ever measure general player skill with perfect accuracy – skill is a subjective concept in the first place you can ultimately define however you want. But there’s no need for perfect accuracy: we only need an objective, and as I mentioned earlier, that objective should be playstyle-neutral. Score suits that purpose much better than the number of kills.

    This is the Internet, so there’s never an end to all arguments, but there’s an enormous difference between A) two players with equal score, and B) two players with equal number of kills.

    2. No incentives for spawn preference

    X representing kill count means that the success of your run is determined largely by the robots that spawn. If you get lots of Reavers and Guardians, you’re screwed, it’s as simple as that. This leads to frustration – and it leads to players restarting and restarting and restarting until they get a favourable spawn. Those players might not even learn to actually deal with the more difficult robots, and they certainly don’t get to appreciate the full richness of the game.

    A score system can measure your performance irrespective of the robots you’re facing. 100k points for killing lots of low-tier bots is equally impressive as 100k for killing a much smaller number of high-tier bots. There’s no real reason to milk spawns beyond pure personal preference.

    Of course, some bots will always perform better in certain maps, so some players will probably still game the system, but that’d still be a huge improvement over the current situation. No robot is going to be 15 times more deadly in one map than on another.

    3. More varied robots with different threat levels

    To this day there has been an incentive for the devs to make all robots roughly similar in their threat level: you only get one kill from each, so it’s unfair for players when one is enormously more powerful than another one. I would argue this has already affected the game, and without such an incentive, the bots we have now would vary more in terms of power and threat level – which would, arguably, lead to a more interesting and fun gameplay experience.

    With a score system there would be no such incentive. Robots could have whatever threat level leads to fun gameplay – we could even have actual bosses in CM – as long as the points you’re rewarded for destroying them are in line with the threat they pose.

    4. Easier balancing of robots

    Point rewards can be adjusted. No need to tone down a robot that’s considered too lethal for the points it provides – just up the points. The primary goal should always be to make the game fun and tactically interesting, and whereas points can always be adjusted to match, rewarding each kill identically works against this goal.

    5. Better single-player campaign

    Huh? Campaign?

    Yes. Single-player campaign, in particular, benefits greatly from having lots of different robots with varying levels of threat. You want both cannon-fodder enemies and serious minibosses and a general upward trend of threat level as the campaign progresses, but as these bots are used both in SP and CM and there’s the aforementioned incentive to keep threat levels in check, the campaign suffers too. As a score system would free the devs from this constrait, we might see more variation and a better difficulty progression in the campaign as well.

    Of course, this is based on the assumption that robots are the same in CM and SP. I can’t emphasize enough how strongly I feel that this should be the case, but that’s a discussion for another thread.

    (6. High Scores!

    Also, Overload is an old-school game in many ways. Wouldn’t it be more fun to see good old High Score listings instead of boring, modern “leaderboards”?)

    *** Why now? Isn’t it too late? ***

    A major part of Early Access is to gather practical experience from players. We’ve been playing for a long time now, and now we have the experience. I’ve been favouring a point system for the past year or so, but the more I play, the more strongly I feel that it would be a major improvement. And it’s still Early Access, it’s not too late.

    I have no experience with Unity, but I’ve been a programmer long enough to say with high confidence that this change would not require a huge amount of work from the devs. Summing point values isn’t all that much different from summing kills and the game already gathers plenty of stats anyway. Also, the statistics gathered by some of the best players would provide an excellent starting point for score values – they’re not going to absolutely perfect, but still much better than what we have now.

    (It should be noted, however, that the stats themselves are likely to be slightly biased by the fact that players tend to avoid spawns with more difficult robots, which leads to players never learning to fight those robots with quite the same proficiency they fight the lesser bots.)

    However, there’s no need to nail down everything perfectly on the first try. I see no reason why robot point rewards couldn’t be fine-tuned long after release, if done with a suitable amount of care.

    Thunderbolt — an elegant weapon, for a more civilized age.

    #15727
    LemurFromTheIdLemurFromTheId
    Kickstarter Backer
    Topics: 3
    Replies: 24

    Ultimately, I think the desire for a point-based scoring system comes from a desire for the challenge mode leaderboards to be competitive in a rigorous way

    I wanted to respond to this specifically.

    I favour a point system, but I’m not a competitive person at all. I don’t compare myself to others. I mostly want a system that’s gives me more accurate feedback on my own performance, because improving my own play is a major source of fun, enjoyment and motivation for me in games like Overload.

    Beyond that, I genuinely believe that a score system would lead to a more fun, varied and tactically interesting game for the vast majority of players.

    Not that it’s a huge issue or anything.

    Thunderbolt — an elegant weapon, for a more civilized age.

    #15728
    PersonicusPersonicus
    Participant
    Topics: 6
    Replies: 35

    Ultimately, I think the desire for a point-based scoring system comes from a desire for the challenge mode leaderboards to be competitive in a rigorous way, but I think this kind of competitiveness is the wrong criterion to optimize for. As long as they list the max score rather than average, pilots who play more will have an advantage over and above their actual skill. The leaderboards are a fun way to provide a goal across runs, but they’re not a serious comparison of pilot skill.

    ^This.

    The suggestion/proposal to introduce a point system smacks of “elitism” in terms of measuring a pilots’ perceived flight skills – it’s an arcade shooter, and the goal is to “get the job done”….it doesn’t have to be pretty. Multi-player will be the true test of pilot skill – no RNG to deal with.

    There would be an alternative way to deal with this: in each difficulty preset, cap the kill count and assign it as a threshold goal for the next difficulty preset (although all presets are open by default). For example (and these are just arbitrary numbers), in Rookie, set the goal at 200 kills. Once achieved, you get the “Rookie medal” and your Rookie scores beyond the capped kill count are no longer shown on the Rookie leaderboard scores. The player may continue to play at the Rookie preset and his/her kill counts will continue to be registered locally, but will receive no further reward in terms of leaderboard kill counts at that preset. The next “medal” would be a cap of 150 kills in Hotshot – the Rookie player having achieved the “Rookie Medal” will now have an incentive to play at a higher difficulty preset to obtain the “Hotshot Medal”.

    To me, this would be an extremely fair way of ensuring that players’ scores are not skewing the leaderboards, would give less experienced players more of a chance of advancing within a given preset and would introduce incentives for pilots to play at higher difficulties. I would hazard a guess that this would be relatively straight forward to implement – the only challenge I foresee, would be in establishing “fair and amicable” kill count caps for each difficulty preset.

    #15730
    LemurFromTheIdLemurFromTheId
    Kickstarter Backer
    Topics: 3
    Replies: 24

    The suggestion/proposal to introduce a point system smacks of “elitism” in terms of measuring a pilots’ perceived flight skills – it’s an arcade shooter, and the goal is to “get the job done”….it doesn’t have to be pretty. Multi-player will be the true test of pilot skill – no RNG to deal with.

    I don’t get this at all. Where does this “elitism” thing come from? I haven’t seen a hint of elitism in anyone I’ve discussed this with.

    The “arcade shooter” argument is an argument for scores. That’s how arcade shooters have always been: you get score for destoying enemies, and you get bigger score for destroying bigger enemies. Play just about any old arcade shooter to confirm.

    There would be an alternative way to deal with this: in each difficulty preset, cap the kill count and assign it as a threshold goal for the next difficulty preset (although all presets are open by default). For example (and these are just arbitrary numbers), in Rookie, set the goal at 200 kills. Once achieved, you get the “Rookie medal” and your Rookie scores beyond the capped kill count are no longer shown on the Rookie leaderboard scores. The player may continue to play at the Rookie preset and his/her kill counts will continue to be registered locally, but will receive no further reward in terms of leaderboard kill counts at that preset. The next “medal” would be a cap of 150 kills in Hotshot – the Rookie player having achieved the “Rookie Medal” will now have an incentive to play at a higher difficulty preset to obtain the “Hotshot Medal”.

    To me, this would be an extremely fair way of ensuring that players’ scores are not skewing the leaderboards, would give less experienced players more of a chance of advancing within a given preset and would introduce incentives for pilots to play at higher difficulties. I would hazard a guess that this would be relatively straight forward to implement – the only challenge I foresee, would be in establishing “fair and amicable” kill count caps for each difficulty preset.

    What does that actually have to do with the subject of score vs. kills? It’s a completely orthogonal idea, you could implement the exact same thing with scores.

    Thunderbolt — an elegant weapon, for a more civilized age.

    #15732
    Luke Schneider
    Overload Team
    Topics: 10
    Replies: 115

    Having a more point-based system is something we discussed early on, and was on the feature list for Early Access, but we just never got around to it. I think I’m generally in favor of it, but I see the appeal and simplicity of the current system as well.

    It’s not a lot of work to change, but it’s not insignificant either. We’ll discuss it as a team in the coming week.

    Luke

    #15734
    YoshimitsuYoshimitsu
    Kickstarter Backer
    Topics: 57
    Replies: 421

    While I’m perfectly happy with the current kill-total scoring, I would also be perfectly happy with a point based system. I think Raptur’s right that the trickiest part will be deciding what value to assign certain Robots. It’s absolutely true that certain ones are way more dangerous on some maps than other. Ogres for instance deal me over 50% extra damage on Foundry because it’s layout favors them. I really need to get my incomplete run data from B94 uploaded…

    I also have come to agree with Raptur that the most accurate way of assessing player skill would be a leaderboard of players’ average scores over all runs per map. I personally don’t find any of this to be actually important since I’m not terribly competitive by nature, but I find keeping an eye on my score and standings to be fun.

    Since I started harvesting data, I’ve actually found all my personal stats against the game to be more interesting than my performance against others. I would love it if Overload itself would eventually track all of these things and give us access to more of our own personal lifetime stats. The new data printing from the last build is awesome (thanks for fixing the bug in the ctrl-a data not labeling bot variants) but integrating it into the game itself would be highly useful and that kind of data from everyone could help inform things like bot balance and what scores to assign them.

    As far as balancing bots for CM impacting the SP difficulty, I can definitely see that as a danger. This newest build seems significantly easier. I’ve said many times that I support bots being balanced for SP first. Keep the same bots in CM but balance their difficulty there by adjusting the spawning rules to limit the numbers of the most dangerous bots WITHOUT EXCEPTION (looking at you Caverns who spawns 8 Reavers in the first 30 bots almost every time).

    Last thought for now; the kills vs. points question doesn’t have to be either or. I would be cool to have both sets of data available on the leaderboard and be able to cycle between them just like we cycle between Countdown and infinite or between different difficulty levels. While we’re at it why not let us sort the boards by player average or jigh score as we choose? I’ve seen capabilities like this in other games (like Halo Reach) and it was always fun and informative. I imagine that getting systems like this functioning for Challenge Mode would also make similar features for Multiplayer easier in the future too.

    #15832
    birdseyebirdseye
    Participant
    Topics: 6
    Replies: 45

    Yinut brings up a good point about robot mixes, IMO you should just have one mix per level, I agree its obvious which mixes are harder and its something I felt like commenting on also. I don’t restart because it feels unfun and makes the game into more of a job. Also, I view CM as more prep for multiplayer someday, rather than something super serious, so I honestly prefer getting the harder robots… but if you are getting hardcore about score, there should be 1 mix.

    You could also have 2 leaderboards, 1 for points and one for total kills, could add fun things to fight over/for.

    #15964
    chibiniviachibinivia
    Participant
    Topics: 4
    Replies: 7

    Ok, I know I’m a month slow on this topic, but this matter is very clear to me. Point system is 100% the way to go.

    Point systems don’t need to be 100% accurate to be a vast improvement over 1 kill = 1 kill. A simple heuristic of bots that do more damage should be worth more points should be good enough. Even if you want a data driven approach, a good looking scoreboard is going to have rounding that will leave it only questionably better then a well implemented heuristic. Points / (damage per kill) ~= constant for all bots should be fairly easy to implement and very intuitive. Going back to Descent 1/2 days, different bots had different point values and it made perfect sense even if you wanted to argue the precise values for any given bot.

    Let’s address a technical issue with adjusting point values and scoreboards. Now it makes sense that once you apply point system to challenge mode that you will want to adjust the point values attributed to each bot occasionally either due to patching or simply the player base evolving. There will be a concern of the validity of the leaderboards across patches. This can be solved by updating the scores on the leaderboards with each points update by having the leaderboards storing the set of kills (e.g. 100 goblins, 50 ogres, etc.) on the back end only displaying the total which would get updated on patch. Yes, some people will lose or gain spots on the leaderboards purely out of patching, but that essentially happens in any competitive game.

    #15965
    krayzkrokkrayzkrok
    Kickstarter Backer
    Topics: 10
    Replies: 185

    I’m also in favour of a point based system, and I think most of the good points have been raised above (particularly relevant are those concerned with the impact a “bad start” can have). I’m not a top rank player by any means, which is why my rank is affected a lot more by the current kill system compared with a points system. Why? Because there are a lot more people who can get 35 kills on Ace on Syrinx than there are people who can get 435 kills on the same map. That means a dozen of us might share the same number of kills, and so the leaderboard ranking can be a bit galling. Although it does mean than an extra kill could bump you up several ranks, it could also go away almost completely by having a scoring system that will provide some distance between these ranks.

Next Unread Topic:
Viewing 12 posts - 1 through 12 (of 12 total)

Forums are currently locked.


Previous Topic: Questions about release date Next Topic: no dead cats & cruel indifference