Any FOSS apps for the Linux CLI that can summarize a large text into short paragraphs or bullet points?
head and tail
Llama2?
Maybe you can use what those tl;dr bots on lemmy use.
Just post it as an article and wait for the bots to show up.
You can use tldr for man pages but for generic text I don’t know. You would probably need a LLM.
could anyone recommend an LLM that could be run locally or on google colab ? thanks
I believe Llama is open source but not sure how complicated it is to get running locally. Nevermind: https://replicate.com/blog/run-llama-locally
You can probably write a bash wrapper around it that feeds in “Can you summarize this text: (text here)” by setting the PROMPT variable the bash script. (Probably just do PROMPT=“Can you summarize this text: $1”) (Obviously don’t recompile everytime so remove the clone build and download code)
thx
Just to warn you it might be very bulky and the model that the script is downloading is deprecated so you’ll have to find a different .gguf model on hugging face. Try to find a lightweight .gguf model and replace the MODEL variable with it nane as well the rest of the link. Or just download from a browser and move it into the models folder.
This is the library one of the Lemmy summarisation bots uses. It can be used as a CLi utility.
PrivateGPT, takes some gigs of storage tho
Not exactly CLI, but GPT4All works perfectly for that.
If you want to have super simple to understand man pages, use “tldr” in the terminal, that’s very helpful!
Might want to look at Llama.cpp.
Check out localAI, an open source drop in replacement for OpenAI APIs, in particular the section about text generation. From here, you can load a model with a specific prompt, in order to summarize text.
