The discussion has moved over to Discord

Home Forums The Archive Overload Early Release Version Brags and Links Build 100 Challenge Mode Data

RSS Feed
Viewing 8 posts - 1 through 8 (of 8 total)
  • Author
    Posts
  • #15827
    YoshimitsuYoshimitsu
    Kickstarter Backer
    Topics: 57
    Replies: 421

    For anyone who feels like it, here’s place to share data that you gather in Build 100.
    I’ll start off with a comparison between my Backfire performances in B94 and B100:
    B94 vs B100

    #15866
    rapturraptur
    Kickstarter Backer
    Topics: 29
    Replies: 134

    I propose a stat for tracking bot difficulty, which we could call its damage index. It is a ratio of two terms that are computed from the stats of a single run. The first term is the proportion of damage received from that bot, as a proportion of the total damage received from all bots. The second term is the kills of that bot as a proportion of the kills in the run. For example, if 10% of the damage a player receives comes from harpies, and 20% of the kills in the run are harpies, then the damage index for harpies in that run is 0.1 / 0.2 = 0.5. The idea is that hard bots deal more damage and provide fewer kills.

    Yoshimitsu was kind enough to share his run stats, so I’ve included histograms of damage index for the two of us in build 100+, broken down by pilot and also collapsed. Notice that the horizontal axis is on a log scale, so a constant distance corresponds to a constant ratio in damage index. We see that Reavers, Tritons, Phantoms, and Guardians consistently have a damage index over 1, while Goblins, Gorgons, Gorgons var, Golems, Harpy var, and Ogres (but not Ogres var) consistently have a damage index under 1. More interestingly, Hydras and Shredders seem to be fairly evenly distributed around 1, but with a larger variance, suggesting that their difficulty is more sensitive to the map and/or bot mix.

    As we accumulate more data, it will be interesting to study how the damage index varies with other run variables.

    –EDIT: these graphs all use Insane difficulty, collapsed across maps.

    View post on imgur.com

    View post on imgur.com

    #15882
    YinutYinut
    Participant
    Topics: 10
    Replies: 55

    I have a very difficult time understanding Raptur’s histograms, while I have zero problems understanding the data presented by Yoshi. I would also like to point out that we already have a very good stat for estimating the threat levels of different robots called “average damage per robot”. As far as Infinite mode is concerned I would say nothing else really needs to even be considered as far as robot power levels go. When it comes to countdown, things get a little more complicated since speed is everything so robot hitpoints start to matter more. Longer kill times = lower kill count.

    #15889
    rapturraptur
    Kickstarter Backer
    Topics: 29
    Replies: 134

    A histogram just divides a measurement into bins, and counts how many runs appear in that bin. For example, you could count how many runs you have where shredders averaged between 20 and 40 damage.

    Unfortunately, we actually cannot compute average damage per bot, however, because we do not know how many bots spawned: we only have how many we killed. Those are not the same number.

    Average damage per bot also has the downside of having an idiosyncratic scale. What counts as a high damage? You have to compare to other bots to set the scale, and the scale could vary by player (some take less damage overall) and run (longer runs have higher damage values because players gather more armor, but then drop as armor spawns decrease and bot spawns increase). Damage index always has an anchor at 1: larger values mean the bot dealt more damage than is justified by the reward, and smaller values dealt less damage than is justified by the reward.

    Damage index is also more useful if we do start scoring by points. The danger/reward trade-off will change, but averaged damage is insensitive to this trade-off. With damage index, however, we can just replace proportion of kills with proportion of points.

    #15890
    YinutYinut
    Participant
    Topics: 10
    Replies: 55

    While I agree with everything you said, I’d say you’re more or less splitting hairs.

    Unfortunately, we actually cannot compute average damage per bot, however, because we do not know how many bots spawned: we only have how many we killed. Those are not the same number.

    While this is true, it would affect the stats to a very small extent, especially over a large sample size. Simple example:
    Kills (krakens) = 100
    Damage (krakens) = 400
    Total krakens spawned = 102
    even if the last 2 krakens dealt 10 dmg each, they would overall affect the averages in a miniscule way. Technically there’s really only very few cases where this is a problem and it’s for example one of those early run ending reaver ambushes where 2 reavers can straight up gank you and through that skew the stats, but again a single run is never considered objective data, hence we need for example 10+ runs at least to start drawing conclusions. The more data we have, the more miniscule this problem gets and I’m using the word problem very loosely here.

    Average damage per bot also has the downside of having an idiosyncratic scale. What counts as a high damage? You have to compare to other bots to set the scale, and the scale could vary by player (some take less damage overall) and run (longer runs have higher damage values because players gather more armor, but then drop as armor spawns decrease and bot spawns increase). Damage index always has an anchor at 1: larger values mean the bot dealt more damage than is justified by the reward, and smaller values dealt less damage than is justified by the reward.

    Well we take the robot with the highest and lowest average damage and use those to set the extremes of the scale, everything else will fall inbetween and will be assigned a relative point-value. Now you’re very much correct that different players get different results, but even so, certain patterns do and already have emerged (ie. guardians are more dangerous than goblins). I’d say the crowd-pleaser approach would be to take data from multiple players and simply run a meta-analysis. The hardcore approach would be to take the stats of the #1 player on the insane leaderboards and simply assess the robot threat levels from the performance of the best player. Also longer runs do not have higher average damage values by robot, only higher total damage values. The only exception to that are situations like the aforementioned Reaver ambush where there barely are any kills on the run and you die prematurely. Also armor drops don’t exactly manipulate the average damage a robot deals.

    #15902
    YoshimitsuYoshimitsu
    Kickstarter Backer
    Topics: 57
    Replies: 421

    I’d say you both make good and accurate points. As far as I’m concerned more ways of seeing data is a good thing so thank you Raptur for producing a new kind of chart! I also had trouble understanding them until you described what histograms are, but the data itself is very interesting. When I’ve got all my runs in I’ll definitely share the data so you can make more use of it with a bigger sample size.

    My opinion on how to determine scores is (as Yinut suggests) to gather as much run data as possible from as many players as possible then use the average damage per killed bot over all runs/players/maps to set the relative scale of robot value. If additional data is needed to influence the point values then Raptur’s histograms seem like a great source.

    I’d be curious, Raptur, to see if you can break down the histograms into a single data point for each bot then compare the scale of those to the scale of damage averages. I’ll bet the order of which bot is the bigger threat will be pretty similar.

    #16229
    YoshimitsuYoshimitsu
    Kickstarter Backer
    Topics: 57
    Replies: 421

    Well, another drop cycle got away from me before I could finish my runs but I did a bit better this time. Here is my data from 5 maps (plus some on Foundry).
    Kills/Damage
    https://www.dropbox.com/s/wl5ydxo2k2ul9id/Yoshimitsu%20Build%20100%2B%20Infinite%20Insane%20combat.ods?dl=0
    Item Drops
    https://www.dropbox.com/s/z332ipt5sewc7oj/Yoshimitsu%20Build%20100%2B%20Infinite%20Insane%20drops.ods?dl=0

    #16231
    YoshimitsuYoshimitsu
    Kickstarter Backer
    Topics: 57
    Replies: 421

    Here we go, a chart of the combined stats from all maps. I arranged the bots by average damage received to show them in ascending order by threat level.
    8.5 Yoshi stats

Next Unread Topic:
Viewing 8 posts - 1 through 8 (of 8 total)

Forums are currently locked.


Previous Topic: Simple CM stat saving tool Next Topic: 0.85 Language issues and borked menu