Lucas Schneider schneider8357

Deploying llm-d on an Air-Gapped Kubernetes Cluster

What is llm-d?

llm-d is a Kubernetes-native distributed inference stack for large language models. It builds on top of vLLM and the Kubernetes Gateway API (plus the Gateway API Inference Extension) to provide smart, model-aware routing, inference pools, and autoscaling primitives — so serving an LLM on Kubernetes feels closer to running a regular web service than hand-rolling vLLM deployments. It's maintained as an incubation project under the llm-d community and is designed to plug into any conformant Gateway API implementation (kgateway, Istio, Envoy Gateway, etc.).

This section was generated by AI to briefly introduce the project — the rest of the guide is based on my hands-on deployment notes.

The overall picture

High Traffic Server Settings on RHEL / AlmaLinux / Rocky / CentOS 8-10

I recently did some work to optimize the network configuration of an AlmaLinux 9 based web server that receives a lot of traffic.

Of course these settings also apply to 8, and I think these should also work on 10 as well but I have not yet tested them.

There is a lot of information on the web for this and it distills down to a minimum recommended configuration, and a recommended configuration.

The minimum recommended configuration should be sufficient for servers with less than 10Gb, and the recommended configuration should be sufficient for servers with 10Gb or more.

Update 28 July 2019: An updated version of this guide for Ubuntu Server 18.04 LTS is now available. Feel free to check it out.

Update 23 May 2020: This guide is ALREADY OUTDATED and might no longer work with new versions of Ubuntu and VirtualBox. Please consider switching to the updated guide instead. I will no longer respond to the replies to this gist. Thank you.

Mounting VirtualBox shared folders on Ubuntu Server 16.04 LTS

This guide will walk you through steps on how to setup a VirtualBox shared folder inside your Ubuntu Server guest. Tested on Ubuntu Server 16.04.3 LTS (Xenial Xerus)

	#!/bin/sh

	# SPDX-FileCopyrightText: 2019 Kirill Elagin <https://kir.elagin.me/>
	# SPDX-License-Identifier: MPL-2.0

	###
	#
	# Monitor signal strength (RSSI) of a Wi-Fi network on Linux.
	#
	# Given an SSID, the script outputs the signal strength of the

	from flask import Flask
	from flask_restful import Api, Resource, reqparse

	app = Flask(__name__)
	api = Api(app)

	users = [
	{
	"name": "Nicholas",
	"age": 42,

	➜ ~ sudo cat /etc/snmp/snmpd.conf
	[sudo] password for mwang:
	###############################################################################
	#
	# snmpd.conf:
	# An example configuration file for configuring the ucd-snmp snmpd agent.
	#
	###############################################################################
	#
	# This file is intended to only be as a starting point. Many more