SadServer solution for https://sadservers.com/scenario/melbourne

Reviewing the code shows me that part Content-Length contains 0 size. So I set the proper size of the code (that is "Hello, World\n" size). And then "sudo systemctl restart gunicorn.service" and "sudo systemctl restart gunicorn.socket" The final code looks like this:

Next issue is with Nginx - inside of "default" config we have file that ends with .socket, and in /run/gunicorn.sock - it use .sock Replace this and restart Nginx "sudo systemctl restart nginx"

Final test:

SadServer - Oaxaca solution for https://sadservers.com/scenario/oaxaca

So, the process is started with the current bash - and it has FD (file descriptor) 77. In the picture, you can see "FD/77w". File descriptor usually usage is: 0 for stdin, 1 for stdout, 2 for stderr, and 255 for bash shell - it points to /dev/tty or /dev/pts/N - where is N number. The main process is bash shell and the subprocess is this FD/77 - so by killing the main process we are doomed to destroy our connections. if we run "lsof somefile" - it will show us our bash shell and under /proc/[PID of shell]/fd/77 we have a symbolic link to /home/admin/somefile.

to release the file descriptor we need to re-use the same number with the command: eval "exec 77>&-"

.bashrc contains the running command:

symbolic link to file:

Unusual task with no points - are you in VM or Docker shell? Simple solution could be this:

SadServer Salt solution URL: https://sadservers.com/scenario/salta

After logging into a server - notice that port 8888 is used. Missing tool lsof, I install with "sudo apt install lsof" and review what process is using port 8888. Nginx was used so I stopped the process with "sudo systemctl stop nginx"

Inside of Dockerfile - found missing proper port 8888 (it was written 8880) and for CMD there was "serve.js" instead of "server.js" - a local file in the same directory.

When the fix was solved, the docker container is built with cmd: "sudo docker build -t sampleapp:v1 . "

running app with "sudo docker run -p 8888:8888 sampleapp:v1" and the task is done

Solution for Cape Town task from URL: https://sadservers.com/scenario/capetown

After logging into the server, there is no working nginx.

Examine details on why nginx does not work show me the first line containing ";". So I removed ";" from the nginx file and nginx does not yet work.

After examining the error log - I was able to see and spot file limits.

After viewing /proc/[pid]/limits - I spot this (Max open files 10)

I did check limits for user www-data, as well as other things (fs.file-max, others)

In the end - Maybe systemD has some limitations per process.

After reading the .service file from systemD

Add # on the start of the line, reload the system daemon, and restart nginx and it works!

For this task, it takes 30 min to solve

The task was to find out how to run a web server on port 80 and to check if is working properly. URL: https://sadservers.com/scenario/tokyo

After checking logs (apache, systems, etc) and finally running iptables - saw HTTP blocked in iptables. So running iptables -F solve this issue (I also added a path for apache /var/www/html/ inside of apache.conf FYI)

SadServer Manhattan - medium, url: https://sadservers.com/scenario/manhattan My first medium task and solution. When I log in - running just

sudo systemctl restart postgresql@14-main.service

The issue is in these lines: "no space left on device"

After running df -h, notice 100% usage for /opt/pgdata/ Removing files that do not need it at all - solved this issue.

Solution

SadServer Apia task url: https://sadservers.com/scenario/apia

Simple file-level task. Using md5sum and diff only.

Compare all files, find unusual md5 hash:

md5sum files

Use diff command, add word in solution, md5sum to check: final commands

SadServer task Bilbao url: https://sadservers.com/scenario/bilbao

After login and inspection of the issue, we get this picture:

Pod status

After googling I found an issue with nodeSelector - so I removed it from the manifest.yml file:

Remove nodeSelector

After removing - it needs to delete all pods, re-run manifest, and check with curl: Delete pods & run yaml file

Solution for Bucharest

Bucharest - url for task: https://sadservers.com/scenario/bucharest File edit:

If we run command:

PGPASSWORD=app1user psql -h 127.0.0.1 -d app1 -U app1user -c '\q'

We could see issue with access and file named "pg_hba.conf " If we open file, and we could see next lines:

host    all             all             all                     reject
host    all             all             all                     reject

If we dig manual for file - we could find that "reject" do that - reject any connection. So replacing with md5 word reject (use sudo for editing file and restarting services), and restart services with

sudo systemctl restart postgresql@13-main.service

LHasa solution for url: https://sadservers.com/scenario/lhasa

In this case we could use awk to sum all numbers in second columns and divide by number of entries ( 100 of them) So 520.4 / 100 -> 5.20

awk '{sum+=$2;} END{print sum;}' scores.txt

Task at: https://sadservers.com/scenario/taipei

Inside of this task we have port knocking, very famous solution to bring protection to access for certain port. More information on this: https://wiki.archlinux.org/title/Port_knocking

One simple way to unlock port 80 is to use nmap 2-3 times on all ports to open port 80:

nmap -Pn -p 1-65535 localhost

Famous port knocking with nmap tool

Solution for: https://sadservers.com/scenario/command-line-murders

File people contain all names, what we need is to find md5 hash of real name to add into mysolution file.

First, command to extract all people names:

awk '{print $1" "$2}' people >> find

After that we need to extract proper name with md5 hash from file "find", so use command:

while read -r col1 ; do
   echo $col1  "$(echo $col1 | md5sum )" | grep 9bba101c7369f49ca890ea96aa242dd5
done < find

and here you go your killer name:

Killer name

Solution for: https://sadservers.com/scenario/santiago

We have 3 files, each contains word "Alice". To count how many there is lines with word Alice

grep Alice *.txt | wc -l

After that we could find file with only once with:

grep Alice *.txt | cut  -d ":" -f 1 | sort | uniq -c | sort -r

Solution for Saskatoon Command:

awk '{print $1}' access.log | sort | uniq -c | sort -r | head -20

With awk, we pickup first field from the line - that is IP address. Second command - make sort this lines so next one command uniq could easy count (that is -c) and sort -r will set reverse list from highest to lower counts of IPs

Count of IPs from access log

For the task https://sadservers.com/scenario/saint-john there is easy solution by usage of tool called lsof (short for list list open files) where man pages https://man7.org/linux/man-pages/man8/lsof.8.html gives details of usage. Take notice here: I use command kill -9, but in case of important tasks like email servers and similar where are data valuable please use kill -15

Solution in one picture:

I'm solving SadServers Challenges!

I've decided to dive into the world of SadServers challenges (https://sadservers.com/scenarios)! This platform offers a variety of system administrator scenarios that test your troubleshooting skills and Linux knowledge.

With over 26 years of experience, I've encountered a wide range of Linux issues, including the infamous "Out of Memory" (OOM) problems with drivers.

Here's the exciting part:

I'll be tackling these challenges and sharing my solutions right here! The first one will be published today, June 7th, 2024, and I'll keep this page updated with links to all my future solutions.

Stay tuned for some in-depth troubleshooting and Linux problem-solving!

Easy

Saint John - Easy - solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-saint-john-task

Saskatoon - Easy solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-saskatoon

Santiago - Easy solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-santiago

Command line murderers - Easy solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-command-line-murderes

Taipei - Easy solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-taipei

Lhasa - Easy math solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-lhasa

Bucharest - Easy, Postgres solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-bucharest

Bilbao - Easy Kubernetes issue solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-biblao-k8s-task

Apia - Easy file tools usage solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-apia-task

Medium

Manhattan - medium, solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-manhattan-medium

Tokyio - medium, solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-tokyio-solution

Cape town - medium, solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-cape-town

Salta - medium, solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-salta-solution

Venice - medium, solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-venice

Oaxaca - medium, solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-oaxaca

Melbourn - medium, solution: https://www.vladimircicovic.com/2024/06/sadserver-solutions-melbourne