A space for technology notes on anything I find interesting, and a musing or two. From electrons through to delivering and developing code, as well as shiny emerging tech.
A brief set of notes on getting Debian-based Linux up and running on an MSI Evo Prestige A13M (028AU) Laptop. The two aspects documented in these notes are making the hard drive available to Linux and getting the WiFi working. These did not work out of the box on first setting up this machine.
This post explores renting a cloud GPU from RunPod and using the vLLM inference engine to run a Large Language Model made available via an OpenAI compatible endpoint, and then load testing that endpoint with K6. What is RunPod? RunPod is a paid cloud GPU provider.
A set of notes on converting a transformers model from Pytorch format to Safetensors format and then quantising to ExLlamaV2 (Exl2) using a code based calibration dataset. This was inspired by posts which reported coding LLMs quantised to Exl2 format using the wikitext default calibration dataset resulted in relatively lower quality outcomes.
A guide to using the Terraform bpg provider to create virtual machines on a Proxmox instance. The bpg provider is a wrapper for the Proxmox API. It enables the provisioning of infrastructure on Proxmox using Terraform. bpg is one of two terraform providers available for Proxmox at time of writing, the other option being telmate.
Copyright © 2024, Alex Darbyshire
Template by Bootstrapious. Ported to Hugo by DevCows.
