Im using Ollama on my server with the WebUI. It has no GPU so its not quick to reply but not too slow either.

Im thinking about removing the VM as i just dont use it, are there any good uses or integrations into other apps that might convince me to keep it?

  • pe1uca@lemmy.pe1uca.dev
    link
    fedilink
    English
    arrow-up
    1
    ·
    3 months ago

    Well, it’s a bit of a pipeline, I use a custom project to have an API to be able to send files or urls to summarize videos.
    With yt-dlp I can get the video and transcribe it with fast whisper (https://github.com/SYSTRAN/faster-whisper), then the transcription is sent to the LLM to actually make the summary.

    I’ve been meaning to publish the code, but it’s embedded in a personal project, so I need to take the time to isolate it '^_^