CubitOom

joined 2 years ago
[–] CubitOom 1 points 8 months ago

Sorry about that bad link. Here it is.

Install ollama

```sh
pacman -S ollama
```

Download any uncensored llm

From ollama's library

Serve, Pull, and Run

  1. In terminal A execute
    ollama serve
    
  2. In terminal B execute
    ollama pull wizard-vicuna-uncensored:7B
    ollama run wizard-vicuna-uncensored:7B
    

From huggingface

download and any gguf model you want with uncensored in the name. I like ggufs from TheBloke

  • Example using SOLAR-10.7B-Instruct-v1.0-uncensored-GGUF
    • Click on Files and Versons and download solar-10.7b-instruct-v1.0-uncensored.Q5_K_S.gguf
    • change directory to where the downloaded gguf is and write a modelfile with just a FROM line
      echo "FROM ~/Documents/ollama/models/solar-10.7b-instruct-v1.0-uncensored.Q5_K_S.gguf" >| ~/Documents/ollama/modelfiles/solar-10.7b-instruct-v1.0-uncensored.Q5_K_S.gguf.modelfile
      
    • Serve, Create, and Run
      1. In terminal A execute
        ollama serve
        
      2. In terminal B execute
        ollama create solar-10:7b -f ~/Documents/ollama/modelfiles/solar-10.7b-instruct-v1.0-uncensored.Q5_K_S.gguf.modelfile
        ollama run solar-10:7b
        

Create a GGUF file from a non gguf llm for ollama

setup python env

Install pyenv and then follow instructions to update .bashrc

curl https://pyenv.run/ | bash

Update pyenv and install a version of python you need

source "${HOME}"/.bashrc
pyenv update
pyenv install 3.9

Create a virtual environment

pyenv virtualenv 3.9 ggufc

Use the virtual environment and download the pre-reqs

pyenv activate ggufc
pip install --upgrade pip
pip install huggingface_hub
mkdir -p ~/Documents/ollama/python
cd ~/Documents/ollama/python
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
pip install -r requirements.txt

Download the model from huggingface.

For this example, Im going to pull llama3.2_1b_2025_uncensored Note that this llm is 1B so can be ran on a low spec device.

mkdir -p ~/Documents/ollama/python
mkdir -p ~/Documents/ollama/models
model_repo_slug='carsenk'
model_repo_name='llama3.2_1b_2025_uncensored'
model_id="$model_repo_slug/$model_repo_name"
cat << EOF >| ~/Documents/ollama/python/fetch.py
from huggingface_hub import snapshot_download

model_id="$model_id"
snapshot_download(repo_id=model_id, local_dir="$model_id",
                  local_dir_use_symlinks=False, revision="main")
EOF

cd ~/Documents/ollama/models
python ~/Documents/ollama/python/fetch.py

Transpose HF to GGUF

python ~/Documents/ollama/python/llama.cpp/convert.py "$model_id" \
  --outfile "$model_repo_name".gguf \
  --outtype q8_0

Serve, Organize, Create, and Run

  1. In terminal A execute
    ollama serve
    
  2. Open a new terminal while ollama is being served.
    mkdir -p ~/Documents/ollama/modelfiles
    echo "FROM ~/Documents/ollama/models/llama3.2_1b_2025_uncensored.gguf" >| ~/Documents/ollama/modelfiles/llama3.2_1b_2025_uncensored.modelfile
    ollama create llama3.2:1b -f ~/Documents/ollama/modelfiles/llama3.2_1b_2025_uncensored.modelfile
    ollama run llama3.2:1b
    
[–] CubitOom 0 points 8 months ago (1 children)

I love kagi but I don't think it actively filters out ai generated content.

I know when searching for pictures you can disable AI generated images.

I think the hard part for a search engine is that unless there is some kind of identifying mark on the content, how do they know that an ai didn't write a top 10 list of pastebin alternatives?

[–] CubitOom 5 points 8 months ago* (last edited 8 months ago) (2 children)

You are right that something that most others will host for free are going to be censored since otherwise they might have some kind of responsibility legally. I learned this while trying to diagnose an issue with my cars door lock.

At the end of the day, anything you ask some hosted llm is being recorded so if you actually want something uncensored or something that gives you a sense of freedom then the only real option is to self host.

Luckily, it's very simple and can even be done on a low spec device if you pick the right model. The amount and type of ram you have a will dictate how many parameters you can run at a decent speed.

Here's 3 options with increasing difficulty (not much however) written for Arch Linux: https://infosec.pub/comment/13623228

[–] CubitOom 2 points 8 months ago

I mean, there's plenty of reasons that someone might want to use something else.

Maybe they just want a linux package to work on android. Also I don't really like how maid asks for nearby device permissions.

[–] CubitOom 3 points 8 months ago (4 children)

Is it possibly a pathing issue?

https://www.yasint.dev/gopath-goroot

PS: I'm pretty interested in what you're trying to do with ollama on android.

[–] CubitOom 1 points 8 months ago

Ok but before I do that, I need to update my emacs config.

[–] CubitOom 6 points 8 months ago (3 children)

organic artisanal buttplug business

OK... I'm interested... Is it like organic rubber? I hope they don't test on animals, although...

[–] CubitOom 1 points 8 months ago

Interesting, I haven't had that experience much myself. It might be a bad port to Linux?

I wonder if there is a launch option that you could set that would help? It might also depend on your GPU and drivers. But to your point, it's much less hassel to just tell steam to use proton and not have those issues.

[–] CubitOom 13 points 8 months ago (6 children)

While I agree that proton on its own doesn't make gaming on Linux a "first class experience", it does sometimes perform better than the original native "first class" Windows OS that the game was originally intended to be played on. Which is just funny, but also shows all the work that has gone into proton.

Game devs need more Linux players before they make major industry wide changes, but proton makes those numbers have a chance of increasing by making the games playable on Linux.

Another reason why I wouldn't call gaming on Linux a "first class experience" yet is controller and input driver issues. Which can be worked around like if I open a game I bought on gog through steam and use the steam input methods but I shouldn't have to use steam to play a gog game with a controller.

[–] CubitOom 2 points 8 months ago (1 children)

Having a restricted form of birthright citizenship is not the same as having no form of it. However there are more countries then I thought that do not have it at all.

At present, 33 countries in the world (and two territories) have unrestricted birthright citizenship, also known as jus soli, and another 32 nations have some form of restricted birthright citizenship

https://worldpopulationreview.com/country-rankings/countries-with-birthright-citizenship

[–] CubitOom 14 points 8 months ago (21 children)

The families - who potentially have young children born in America - will be told to decide for themselves whether to exit the country together or be broken up.

That sounds like a weird way to say that they will be deporting US citizens.

[–] CubitOom 2 points 8 months ago

This song is great. I also love the cover by Elliott Smith.

view more: ‹ prev next ›