Any FOSS apps for the Linux CLI that can summarize a large text into short paragraphs or bullet points?

  • zoe @infosec.pub
    link
    fedilink
    arrow-up
    2
    arrow-down
    1
    ·
    1 year ago

    could anyone recommend an LLM that could be run locally or on google colab ? thanks

    • 257m@lemmy.ml
      link
      fedilink
      arrow-up
      5
      ·
      edit-2
      1 year ago

      I believe Llama is open source but not sure how complicated it is to get running locally. Nevermind: https://replicate.com/blog/run-llama-locally

      You can probably write a bash wrapper around it that feeds in “Can you summarize this text: (text here)” by setting the PROMPT variable the bash script. (Probably just do PROMPT=“Can you summarize this text: $1”) (Obviously don’t recompile everytime so remove the clone build and download code)

        • 257m@lemmy.ml
          link
          fedilink
          arrow-up
          1
          ·
          edit-2
          1 year ago

          Just to warn you it might be very bulky and the model that the script is downloading is deprecated so you’ll have to find a different .gguf model on hugging face. Try to find a lightweight .gguf model and replace the MODEL variable with it nane as well the rest of the link. Or just download from a browser and move it into the models folder.