Introduction
When a distributed denial-of-service (DDoS) attack strikes your Ubuntu servers, every minute of downtime costs you reputation, revenue, and user trust. The recent prolonged outage of Canonical's own web infrastructure—lasting over 24 hours and linked to a pro-Iran group using a "Beam" stressor—shows how even industry leaders can be brought to their knees. This guide walks you through the essential steps to detect, communicate, and mitigate a DDoS attack against Ubuntu-based systems, drawing lessons from that real-world incident. By following these steps, you can minimize damage, maintain transparency, and restore services as quickly as possible.
What You Need
- Access to your Ubuntu server's command line (SSH) with root or sudo privileges
- A network monitoring tool (e.g., tcpdump, ntop, or a cloud provider's dashboard)
- A status page or communication channel (e.g., Twitter, a dedicated status subdomain)
- Contact information for your hosting provider or upstream network operator
- A DDoS mitigation service subscription (e.g., Cloudflare, AWS Shield, or a scrubbing center)
- A disaster recovery plan (written and rehearsed)
- Backup configurations and snapshots of your server state
Step-by-Step Guide
Step 1: Confirm the Attack and Assess Impact
If your webpages, update repositories, or other services become unreachable, immediately verify whether it's a DDoS. Check your server's network traffic using tools like iftop or nload. Look for abnormally high incoming bandwidth or a flood of requests from diverse IP addresses. In the Canonical outage, attempts to connect to Ubuntu and Canonical webpages failed for over 24 hours, while mirror sites continued to work—a strong indicator that the core infrastructure, not the entire internet, was targeted. Also consult your cloud provider's monitoring console for traffic anomalies. Document the start time, affected services (proceed to Step 2 once confirmed).
Step 2: Communicate Internally and Externally
Silence can be misinterpreted as incompetence. As Canonical learned, radio silence after the initial outage only fueled speculation. Establish a clear communication chain: notify your internal incident response team, then publish a brief status message on your designated status page and social media. A sample message: "We are aware of an ongoing attack on our infrastructure. Services may be intermittent. We are working to restore full functionality. Updates will follow here." Use a status page that leverages a different hosting provider or a static site on a CDN to remain accessible even when your main servers are down. The Canonical status page did stay up, which is a good practice.
Step 3: Engage Mitigation Services Immediately
Do not try to fight a large-scale DDoS alone. Contact your hosting provider or cloud vendor to enable DDoS protection (e.g., activate AWS Shield Advanced or Cloudflare's "Under Attack" mode). Redirect traffic through a scrubbing center that filters malicious packets. If you have a third-party mitigation service, activate it now. Update your DNS records to point to the mitigation service's IPs. This step mirrors what any organization should do when faced with a "sustained, cross-border attack" as reported by Canonical.
Step 4: Apply Temporary Server-Level Filters
While awaiting upstream filtering, you can reduce load on your servers by applying temporary rules. Use iptables or nftables to rate-limit incoming connections per IP, block suspicious ports, or drop packets from known bad actors. For example: iptables -A INPUT -p tcp --dport 80 -m connlimit --connlimit-above 20 -j DROP. If your web application uses a reverse proxy (like Nginx), enable request rate limiting at the application layer. Be careful not to block legitimate users—test rules on a non-production server first.
Step 5: Monitor and Adapt
DDoS attacks can change patterns over time. Continuously monitor traffic after mitigation is deployed. Check logs and traffic graphs every 15-30 minutes. The same pro-Iran group that targeted Canonical also claimed attacks on eBay, indicating they may rotate targets and methods. If you see new attack vectors (e.g., an increase in DNS amplification), update your filters accordingly. Keep your communication channel updated with status changes: "Attack still ongoing, but mitigation is stabilizing traffic."
Step 6: Preserve Evidence
After the attack subsides, collect logs, pcap files, and incident reports. This evidence may be useful for legal action, sharing with law enforcement, or post-mortem analysis. The group behind the Canonical attack used a "Beam" stressor—documenting such details can help security researchers. Save copies of mitigation steps taken, timestamps, and affected services. This will be invaluable for improving your response plan.
Step 7: Restore Full Services and Conduct Post-Mortem
When traffic returns to normal levels, gradually remove temporary filters. Test each service (web, update repository, login) before declaring all-clear. Then schedule a post-mortem meeting within 48 hours. Discuss what worked, what didn't, and update your incident response plan. The Canonical incident lasted over a day; consider what additional measures (like load balancing across multiple data centers) could shorten future outages.
Tips for a Robust DDoS Response
- Never assume it won't happen to you. Even infrastructure giants like Canonical were caught off guard. Always have a DDoS response plan ready.
- Keep your status page on a separate, hardened platform. Use a static site hosted on a CDN that is not part of your main infrastructure.
- Practice incident communication. Draft templates for different phases of an attack—initial notification, ongoing updates, and recovery announcement.
- Diversify your infrastructure. Use multiple upstream providers and IP ranges so a single attack can't take down everything at once.
- Regularly test your mitigation tools. Run tabletop exercises that simulate a DDoS scenario involving your team.
- Monitor threat intelligence feeds. Be aware of active attacker groups, such as pro-Iranian hackers, and their preferred tools (e.g., "Beam" stressor).
- Back up your data and configurations redundantly. In the event of prolonged downtime, you may need to rebuild from scratch.
- Consider a DDoS insurance policy to cover lost revenue and recovery costs.
By following these steps and tips, you can navigate a DDoS attack on Ubuntu infrastructure with clarity and resilience—exactly what Canonical's team is working to do as this incident unfolds.