Systemd Service Setup - UltraBalancer

Overview

Run UltraBalancer as a systemd service for automatic startup, restart on failure, and easy management on Linux systems.

Auto-start

Start automatically on boot

Auto-restart

Restart on failure automatically

Log Management

Integration with journald

Resource Limits

Control CPU and memory usage

Basic Service Setup

Create Service File

sudo tee /etc/systemd/system/ultrabalancer.service << 'EOF'
[Unit]
Description=UltraBalancer Load Balancer
Documentation=https://docs.ultrabalancer.com
After=network.target

[Service]
Type=simple
User=ultrabalancer
Group=ultrabalancer
ExecStart=/usr/local/bin/ultrabalancer --config /etc/ultrabalancer/config.yaml
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
EOF

Create User and Directories

# Create system user
sudo useradd -r -s /bin/false ultrabalancer

# Create directories
sudo mkdir -p /etc/ultrabalancer
sudo mkdir -p /var/log/ultrabalancer

# Set ownership
sudo chown -R ultrabalancer:ultrabalancer /etc/ultrabalancer
sudo chown -R ultrabalancer:ultrabalancer /var/log/ultrabalancer

Enable and Start Service

# Reload systemd
sudo systemctl daemon-reload

# Enable service (start on boot)
sudo systemctl enable ultrabalancer

# Start service
sudo systemctl start ultrabalancer

# Check status
sudo systemctl status ultrabalancer

Production Service Configuration

/etc/systemd/system/ultrabalancer.service

[Unit]
Description=UltraBalancer High-Performance Load Balancer
Documentation=https://docs.ultrabalancer.com
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=ultrabalancer
Group=ultrabalancer

# Main service
ExecStart=/usr/local/bin/ultrabalancer --config /etc/ultrabalancer/config.yaml
ExecStartPre=/usr/local/bin/ultrabalancer validate --config /etc/ultrabalancer/config.yaml
ExecReload=/bin/kill -HUP $MAINPID
ExecStop=/bin/kill -TERM $MAINPID

# Restart policy
Restart=always
RestartSec=10s
StartLimitInterval=60s
StartLimitBurst=3

# Security
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/log/ultrabalancer
CapabilityBoundingSet=CAP_NET_BIND_SERVICE

# Resource limits
LimitNOFILE=65536
LimitNPROC=4096
MemoryLimit=2G
CPUQuota=200%

# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=ultrabalancer

# Working directory
WorkingDirectory=/var/lib/ultrabalancer

[Install]
WantedBy=multi-user.target

Service Management

Common Commands

# Start service
sudo systemctl start ultrabalancer

# Stop service
sudo systemctl stop ultrabalancer

# Restart service
sudo systemctl restart ultrabalancer

# Reload configuration (no downtime)
sudo systemctl reload ultrabalancer

Graceful Reload

# Reload configuration without dropping connections
sudo systemctl reload ultrabalancer

# Or send SIGHUP directly
sudo kill -HUP $(systemctl show -p MainPID ultrabalancer | cut -d'=' -f2)

Advanced Configurations

With Socket Activation

/etc/systemd/system/ultrabalancer.socket

[Unit]
Description=UltraBalancer Socket
PartOf=ultrabalancer.service

[Socket]
ListenStream=0.0.0.0:80
ListenStream=[::]:80
Accept=false

[Install]
WantedBy=sockets.target

/etc/systemd/system/ultrabalancer.service

[Unit]
Description=UltraBalancer Load Balancer
Requires=ultrabalancer.socket
After=ultrabalancer.socket

[Service]
Type=notify
ExecStart=/usr/local/bin/ultrabalancer --config /etc/ultrabalancer/config.yaml --systemd-socket
Sockets=ultrabalancer.socket

[Install]
WantedBy=multi-user.target

# Enable socket activation
sudo systemctl enable ultrabalancer.socket
sudo systemctl start ultrabalancer.socket

Multiple Instances

/etc/systemd/system/[email protected]

[Unit]
Description=UltraBalancer Instance %i
After=network.target

[Service]
Type=simple
User=ultrabalancer
Group=ultrabalancer
ExecStart=/usr/local/bin/ultrabalancer --config /etc/ultrabalancer/config-%i.yaml
Restart=always

[Install]
WantedBy=multi-user.target

# Start multiple instances
sudo systemctl start ultrabalancer@api
sudo systemctl start ultrabalancer@web
sudo systemctl start ultrabalancer@internal

# Check all instances
sudo systemctl status ultrabalancer@*

With Environment File

# Create environment file
sudo tee /etc/ultrabalancer/ultrabalancer.env << 'EOF'
ULTRA_LOG_LEVEL=info
ULTRA_LOG_FORMAT=json
ULTRA_WORKERS=8
ULTRA_MAX_CONNECTIONS=50000
EOF

/etc/systemd/system/ultrabalancer.service

[Service]
EnvironmentFile=/etc/ultrabalancer/ultrabalancer.env
ExecStart=/usr/local/bin/ultrabalancer --config /etc/ultrabalancer/config.yaml

Health Check Integration

/etc/systemd/system/ultrabalancer.service

[Service]
Type=notify
ExecStart=/usr/local/bin/ultrabalancer --config /etc/ultrabalancer/config.yaml
ExecStartPost=/usr/local/bin/ultrabalancer-health-check.sh
Restart=on-failure
RestartSec=5s

# Watchdog
WatchdogSec=30s

/usr/local/bin/ultrabalancer-health-check.sh

#!/bin/bash
# Wait for service to start
sleep 5

# Check health endpoint
if ! curl -f http://localhost:8080/metrics > /dev/null 2>&1; then
    echo "Health check failed"
    exit 1
fi

echo "Health check passed"
exit 0

Resource Limits

CPU Limits

[Service]
# Limit to 200% CPU (2 cores)
CPUQuota=200%

# Weight for CPU scheduling (default 1024)
CPUWeight=1024

# CPU affinity
CPUAffinity=0-7

Memory Limits

[Service]
# Hard memory limit
MemoryLimit=2G

# Memory high watermark (throttle if exceeded)
MemoryHigh=1.5G

# Memory maximum (OOM kill if exceeded)
MemoryMax=2G

File Descriptor Limits

[Service]
# Increase file descriptor limit
LimitNOFILE=65536

# Process limit
LimitNPROC=4096

# Core dump size
LimitCORE=infinity

Security Hardening

Restrictive Service

[Service]
# User and group
User=ultrabalancer
Group=ultrabalancer
DynamicUser=false

# Security
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_INET AF_INET6
RestrictNamespaces=true
RestrictRealtime=true
RestrictSUIDSGID=true
LockPersonality=true

# Filesystem
ReadWritePaths=/var/log/ultrabalancer
ReadOnlyPaths=/etc/ultrabalancer

# Capabilities
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE

# System calls
SystemCallFilter=@system-service
SystemCallFilter=~@privileged @resources
SystemCallErrorNumber=EPERM

With SELinux

# Create SELinux policy
sudo semanage port -a -t http_port_t -p tcp 8080
sudo setsebool -P httpd_can_network_connect 1

# Allow systemd to manage service
sudo semanage fcontext -a -t bin_t "/usr/local/bin/ultrabalancer"
sudo restorecon -v /usr/local/bin/ultrabalancer

Monitoring and Alerting

Systemd Status Notifications

[Service]
# Notify systemd of startup completion
Type=notify
NotifyAccess=main

# Send watchdog keepalive
WatchdogSec=30s

Email Notifications on Failure

[Unit]
OnFailure=ultrabalancer-failure@%n.service

[Service]
# ... existing service config ...

/etc/systemd/system/[email protected]

[Unit]
Description=Send failure notification for %i

[Service]
Type=oneshot
ExecStart=/usr/local/bin/send-alert.sh %i

/usr/local/bin/send-alert.sh

#!/bin/bash
SERVICE=$1

echo "Service $SERVICE has failed!" | \
    mail -s "Alert: Service Failure" [email protected]

Integration with Monitoring

# Expose systemd status to Prometheus
# Install node_exporter with systemd collector
sudo apt install prometheus-node-exporter

# Metrics available at:
# http://localhost:9100/metrics
# systemd_unit_state{name="ultrabalancer.service"}

Log Management

Journald Configuration

/etc/systemd/journald.conf

[Journal]
Storage=persistent
Compress=yes
MaxRetentionSec=1month
MaxFileSec=1week

# View logs
sudo journalctl -u ultrabalancer

# JSON output
sudo journalctl -u ultrabalancer -o json

# Export logs
sudo journalctl -u ultrabalancer > ultrabalancer.log

# Clear old logs
sudo journalctl --vacuum-time=7d

Forward to Syslog

[Service]
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=ultrabalancer
SyslogFacility=daemon

Log to File

[Service]
StandardOutput=append:/var/log/ultrabalancer/access.log
StandardError=append:/var/log/ultrabalancer/error.log

Troubleshooting

Service fails to start

# Check service status
sudo systemctl status ultrabalancer

# View detailed logs
sudo journalctl -xe -u ultrabalancer

# Validate configuration
sudo ultrabalancer validate -c /etc/ultrabalancer/config.yaml

# Check file permissions
ls -la /usr/local/bin/ultrabalancer
ls -la /etc/ultrabalancer/config.yaml

Permission denied errors

# Check user exists
id ultrabalancer

# Check file ownership
sudo chown ultrabalancer:ultrabalancer /etc/ultrabalancer/config.yaml
sudo chown ultrabalancer:ultrabalancer /var/log/ultrabalancer

# Grant capability for privileged ports
sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/ultrabalancer

Service keeps restarting

# Check restart count
systemctl show ultrabalancer | grep NRestarts

# Disable automatic restart temporarily
sudo systemctl edit ultrabalancer
# Add:
[Service]
Restart=no

# Reload and debug
sudo systemctl daemon-reload
sudo systemctl restart ultrabalancer
sudo journalctl -u ultrabalancer -f

Cannot reload configuration

# Validate config first
sudo ultrabalancer validate -c /etc/ultrabalancer/config.yaml

# Check if SIGHUP is handled
sudo kill -HUP $(systemctl show -p MainPID ultrabalancer | cut -d'=' -f2)

# If reload doesn't work, restart
sudo systemctl restart ultrabalancer

Best Practices

Production Recommendations

Always validate config before reload

sudo ultrabalancer validate -c /etc/ultrabalancer/config.yaml && \
sudo systemctl reload ultrabalancer

Use RestartSec to prevent rapid restarts

RestartSec=10s
StartLimitInterval=60s
StartLimitBurst=3

Enable service after testing

sudo systemctl start ultrabalancer
# Test thoroughly
sudo systemctl enable ultrabalancer

Monitor journald logs regularly

sudo journalctl -u ultrabalancer --since "1 hour ago"

Set appropriate resource limits

LimitNOFILE=65536
MemoryLimit=2G
CPUQuota=200%

Linux Server

Deploy on bare metal Linux servers

Docker

Deploy with Docker containers

Monitoring

Set up monitoring and alerts

Troubleshooting

Debug production issues

Getting Started

Concepts

Configuration

Deployment

Testing

API

Advanced

​Overview

Auto-start

Auto-restart

Log Management

Resource Limits

​Basic Service Setup

​Create Service File

​Create User and Directories

​Enable and Start Service

​Production Service Configuration

​Service Management

​Common Commands

​Graceful Reload

​Advanced Configurations

​With Socket Activation

​Multiple Instances

​With Environment File

​Health Check Integration

​Resource Limits

​CPU Limits

​Memory Limits

​File Descriptor Limits

​Security Hardening

​Restrictive Service

​With SELinux

​Monitoring and Alerting

​Systemd Status Notifications

​Email Notifications on Failure

​Integration with Monitoring

​Log Management

​Journald Configuration

​Forward to Syslog

​Log to File

​Troubleshooting

​Best Practices

​Related Topics

Linux Server

Docker

Monitoring

Troubleshooting

Overview

Basic Service Setup

Create Service File

Create User and Directories

Enable and Start Service

Production Service Configuration

Service Management

Common Commands

Graceful Reload

Advanced Configurations

With Socket Activation

Multiple Instances

With Environment File

Health Check Integration

Resource Limits

CPU Limits

Memory Limits

File Descriptor Limits

Security Hardening

Restrictive Service

With SELinux

Monitoring and Alerting

Systemd Status Notifications

Email Notifications on Failure

Integration with Monitoring

Log Management

Journald Configuration

Forward to Syslog

Log to File

Troubleshooting

Best Practices

Related Topics