• FizzyOrange@programming.dev
    link
    fedilink
    arrow-up
    0
    ·
    5 days ago

    Private or obscure ones I guess.

    Real-world (macro) benchmarks are at least harder to game, e.g. how long does it take to launch chrome and open Gmail? That’s actually a useful task so if you speed it up, great!

    Also these benchmarks are particularly easy to game because it’s the actual benchmark itself that gets gamed (i.e. the code for each language); not the thing you are trying to measure with the benchmark (the compilers). Usually the benchmark is fixed and it’s the targets that contort themselves to it, which is at least a little harder.

    For example some of the benchmarks for language X literally just call into C libraries to do the work.