• T156@lemmy.world
    link
    fedilink
    English
    arrow-up
    7
    ·
    15 hours ago

    The cost has also shot up because a lot of the new frameworks are much more token heavy than the old ones.

    So the original free plan might have made sense when people were only typing little questions into it, and using a handful of tokens, but is no longer cost-effective with things like modern agent pipelines constantly throwing tens of thousands of tokens at the service.

    I tried running a little locally hosted agent thing on my computer the other day, and it was feeding a hundred thousand tokens at the model every few minutes, because it was keeping all the files in context. Sure, it hit the cache a lot, and so the effective cost would be less, but it’s still a lot more token usage than me poking the model with inane questions.