I thought that installing NVIDIA RTX A6000 ADA in default Ubuntu 22 server installation would be an easy one. However, installing drivers from the repository made no good. I verified if secure boot is enable and no it was disabled. We need to install few things first: We need to get rig of previously installed drivers: Verify if secure boot is disabled: Get NVIDIA driver, such as NVIDIA-Linux-x86_64-535.216.01.run from their webiste and install it: In case you got rid of previously installed drivers, disabled secure boot and installed build tools, kernel headers… you will be good to go to compile
LLMs contain built-in policies for protecting minors, animals etc. Monkey eating sausage should be against policy. But it can be fooled and finally model stops complaining and describe what we want to. Tried: to generate funny/controversial pictures. Actuall image generate takes place at Stable Diffusion and not at those conversational LLMs. However, once aksed to generate something dubious or funny they tend to reject such requests hiding befind their policies. Refusals from nexusraven and granite3-dense First I asked for Proboscis Monkey holding can of beer and eating sausage. LLM model called nexusraven refused with that request: nexusraven: I cannot fulfill
Generate WebGL, Three.JS, HTML, CSS, JavaScript, no Python code, single page with rotating cube, ambient lighting. Load libraries from CDN. Let ambient lighting be as such cube edges are visible. add directional lighting also pointing at the cube. Scene needs to be navigable using arrow keys. Ensure browser compability. With codegemma:7b you can generate source code. If asked properly then in WebUI chat a artifacts feature will appear, interpreting your source code immediately, just after source code is generated. This feature is useful for designers, developers and marketers who would like to speed-up scaffolding and migrating from brainstorm into visible
My local toolbox was empty, now it’s full. Lately I have been writing about Ollama, WebUI and StableDiffusion on top of Automatic1111. I found myself struggling a little bit to keep up with all those information about how to run it in specific environments. So here you have an extract of step by step installation. Starting with NVIDIA driver and some basic requirements: Next we go for Docker. Ollama, but with binaries instead of Docker container. It will be much easier, and does not require installing Docker extensions for GPU acceleration support: If running Ollama on different server, then need
I want to create custom chatbot experience. I want to be based on Google’s Gemma AI Large Language Models. I find Gemma3, especially 27b version very capable while problem solving. It has been trained on such data that I find it interesting. I will use Open WebUI to create custom “model hat” and provide chatbot experience TLDR In order to create your own chatbot, only 3 steps are required: To create own chatbot experience I can use System Prompt feature which is core part of model itself. Running on Ollama, Gemma3:27b is actually a 4-bit quantized version of full 16-bit
slight utlization drop when dealing with multi GPU setup TLDR Power usage and GPU utilization varies between single GPU models and multi GPU models. Deal with it. My latest finding is that single GPU load in Ollama/Gemma or Automatic1111/StableDiffusion is higher than using multiple GPUs load with Ollama when model does not fit into one GPU’s memory. Take a look. GPU utilization of Stable Diffusion is at 100% with 90 – 100% fan speed and temperature over 80 degress C. Compare this to load spread across two GPUs. You can clearly see that GPU utilization is much lower, as well
With Ollama paired with Gemma3 model, Open WebUI with RAG and search capabilities and finally Automatic1111 running Stable Diffusion you can have quite complete set of AI features at home in a price of 2 consumer grade GPUs and some home electricity. With 500 iterations and image size of 512×256 it took around a minute to generate response. I find it funny to be able to generate images with AI techniques. Tried Stable Diffusion in the past, but now with help of Gemma and integratino with Automatic1111 on WebUI, it’s damn easy. Step by step Prerequisites You can find information
Utilize both CPU, RAM and GPU computational resources With Ollama you can use not only GPU but also CPU with regular RAM go run LLM models, like DeepSeek-R1:70b. Of course you need to have fast both CPU and RAM and have plenty of it. My Lab setup contains 24 vCPU (2 x 6 cores * 2 threads) and from 128 to 384 GB of RAM. Once started, Ollama allocates 22.4GB in RAM (RES) and 119GB of vritual memory. It occupies 1200% CPU utilization causing system load to go up to 12. However, CPU utilization is only 50% in total. It
Ollama with WebUI on 2 “powerful” GPUs feels like commercial GPTs online I thought that Exo would do the job and utilize both of my Lab servers. Unfortunately, it does not work on Linux/NVIDIA with my setup and following official documentation. So I went back to Ollama and I found it great. I have 2 x NVIDIA RTX 3060 with 12GB VRAM each giving me in total 24GB which can run Gemma3:27b or DeepSeek-r1:32b. Ollama can utilize both GPUs in my system which can be seen in nvidia-smi. How to run Ollama in Docker with GPU acceleration you can read