the end result: an LLM from Huggingface hub running locally with ollama backend prerequisites: an Ubuntu/Debian VM with NVIDIA GPU, conda, git etc. not sure if a GPU is needed for this pa...
I put together a very flexible, straightforward and scalable workflow to share GPU compute and VRAM resources between LXCs and VMs. So here are my notes on how to share a GPU (or multiple GPUs) be...
My notes from last year (2024) on how I set up a lenovo thinkbook gen6 laptop with ubuntu 24.04 as a hypervisor for QEMU/KVM virtualization with GPU passthrough to guests and hibernation. end resu...
Here I go through the local services I run on my GPU rig and how I set them up. Namely, Docker containers with Homepage, Jellyfin, hashcat and IT-Tools The video on Youtube Github repo My GPU ...
How to build a minimalistic GPU server with 24GB VRAM for running inference and training using modern CUDA. My video on Youtube Github repo GPU home server
End result Ollama instance with open-sourced LLM weights running on a local machine (or any machine on the local network for that matter), accessible via open-webui or Python API on the same ma...
The end result 2 pools with 4+2 erasure coding, slow nearline storage on spinning rust and fast solid state storage for model training. Each pool has its own CEPH FS (could be done with a singl...
Reworked the website from React/Vite to Jekyll with Chirpy theme, mainly following this tutorial https://technotim.live/posts/jekyll-docs-site/. The old website was a quick fix with React/Vite from...