Simple Web Service Architecture for Personal Use

Sometimes, we have time and cost limitations when developing a personal project. As a developer, hesitation will arise when deciding to follow a monolithic or micro-services pattern before initiating the project. Of course, everyone will suggest choosing a monolithic pattern because it is the smallest and simplest solution. However, the best solution for me is simple separation, as shown below.

System Architecture | Created with draw.io

At first glance, you might think it looks a bit weird because it has separation like a micro-service pattern, but all of them run in the same VM, like a monolith pattern. But it's worth knowing that it gives the best trade-off between scalability and simplicity. Adopting a pure monolithic pattern would sacrifice flexibility for future development and make it difficult to utilise extra resources. However, fully adopting a micro-service pattern would require additional time and cost in the MVP phase because orchestration tools such as Kubernetes or Docker are needed.

That simple separation also allowed me to explore different languages and frameworks. The FE Service uses pure JavaScript with the Next.js framework because I want to explore its SSG and SSR capabilities. The BE Service use the Golang language because I am curious about the claim for its robustness  and lightweight runtime. I try to use Typescript and Express.js framework for the Auth Service since Full-stack developers often use it. Of course, there's a downside in VM computing and memory usage if we use different languages because they need different runtimes, which produce more overhead.

Currently, the DB is installed in the same VM with the three services above to reduce the extra cost for provisioning dedicated persistence storage. As you know, that is a bad design because the chances of data loss are higher. To compensate for that risk, there's an extra script to back up the DB data and periodically send it to other storage outside the VM. In my case, I use Google Drive as the backup destination for cost efficiency. I will explain that backup job in a different article. 

Below is the step-by-step guidance for implementing the above architecture. Please be aware that some commands or detailed steps may be obsolete when you read them. Then, check other sources and create a VM snapshot as a backup and checkpoint each time you complete certain steps smoothly without errors or problems.

1) Create & Setup VM

If you want to fully follow the subsequent step, I recommend you choose Debian Linux for the OS. Just follow the common installation step until the terminal appears, and you can connect to VM via SSH

2) Setup & Deploy The Services

In my architecture, there are three separate services. But all inbound traffic will be handled by the Next.js Web Service/FE Service. Then, the FE Service will communicate internally with Golang BE Service and Auth Service. Just browse for the details steps for deploying your own service since different languages and frameworks might have different steps. Then continue to the next step below after you can access your FE/Web Service via browser with your VM public IP + port that you set for runtime (the common default port is 8080), for example, http://10.123.127.2:8080

3) Install & Set Up Reverse Proxy NGINX

If you buy a domain, you must set up DNS routing to your VM IP. Then, you can access your website with the domain, but still use a port number. Nginx is a reverse proxy that routes incoming traffic to the specific port based on the domain. So, the user can access our service without specifying the port number like a common website on the Internet. To install Nginx on your machine, execute the following command.

sudo apt install nginx

Then register your domain to the sites-available as shown below using your favourite text editor (skip the line with comment # managed by Certbot because it’s automatically created when applying SSL with Certbot at the next step)

sudo nano /etc/nginx/sites-available/yourdomain.my.id

# Virtual Host configuration for example.com
#
# You can move that to a different file under sites-available/ and symlink that
# to sites-enabled/ to enable it.
#

server {
	server_name yourdomain.my.id;

   #root /var/www/html/apple;
   #index index.html;

   location / {
			#try_files $uri $uri/ =404;
      proxy_pass http://localhost:3000;
      proxy_set_header Host $host;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header X-Forwarded-For $remote_addr;

      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "Upgrade";
      proxy_set_header X-Real-IP $remote_addr;

      proxy_hide_header x-middleware-rewrite;
 
      # Increase buffer sizes to handle large headers
      proxy_buffers 16 64k;            # 16 buffers of 64KB (Total: 1MB)
      proxy_buffer_size 128k;          # Handle large headers
      proxy_busy_buffers_size 128k;    # For active responses
      proxy_temp_file_write_size 128k; # Max temp file size
   }

   listen [::]:443 ssl ipv6only=on; # managed by Certbot
   listen 443 ssl; # managed by Certbot
   ssl_certificate /etc/letsencrypt/live/yourdomain.my.id/fullchain.pem; # managed by Certbot
   ssl_certificate_key /etc/letsencrypt/live/yourdomain.my.id/privkey.pem; # managed by Certbot
   include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
   ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
	if ($host = yourdomain.my.id) {
		return 301 https://$host$request_uri;
  } # managed by Certbot
 
   listen 80;
   listen [::]:80;

   server_name yourdomain.my.id;
   return 404; # managed by Certbot
}

After saving the content, execute the below command to validate if there’s any error

sudo nginx -t

if no error, then execute the below command to create symlink for enabling the routing

sudo ln -s /etc/nginx/sites-available/yourdomain.my.id /etc/nginx/sites-enabled/

Now, you can access your website without specifying the port number. However, your website is still not secure because the data transferred is not encrypted yet with the HTTPS protocol.

4) Install CertBot for SSL

To secure your website by restricting access to HTTPS only , you can apply free SSL to NGINX with commands below

sudo apt install certbot python3-certbot-nginx
sudo certbot --nginx -d yourdomain.my.id
sudo systemctl restart nginx

5) Install MySQL

Below are the steps to install DB that I use

wget https://dev.mysql.com/get/mysql-apt-config_0.8.22-1_all.deb
sudo dpkg -i mysql-apt-config_0.8.22-1_all.deb
sudo apt install mysql-server
sudo service mysql status
sudo dpkg-reconfigure mysql-apt-config
sudo apt update
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys B7B3B788A8D3785C
sudo apt update
sudo apt install mysql-server
sudo service mysql status

6) Set Up MySQL User & DB

The user and the DB are separated for each service to follow micro-service best practices. Below are some steps that I use

select * from mysql.user;

CREATE USER 'service_a@'localhost' IDENTIFIED BY 'yourpassword';
create database service_a;
GRANT ALL PRIVILEGES on service_a.* TO ' service_a @'localhost';
FLUSH PRIVILEGES;

Then, iterate the above process for the other services

7) Set Up SystemD

I use SystemD to handle my services' background runtime because it's the default tool in Linux. I don't need to install anything else, and it's pretty simple to set up for my requirements. 

To register the runtime to systemD, you need to create a file below in the path with the pattern /etc/systemd/system/your-service.service. Below is a sample of the file content. The content may vary depending on the language and framework you use.

GNU nano 5.4      /etc/systemd/system/backend-service.service   
[Unit]
Description=Backend Service
After=network.target

[Service]
# Define the user and group that will run the service
User=youruser
Group=yourgroup

# Set the working directory to the app directory
WorkingDirectory=/home/youruser/backend-service

# Path to the Go application binary
ExecStart=/usr/local/go/bin/go run /home/youruser/backend-service

# Restart the service on failure
Restart=on-failure

[Install]
WantedBy=multi-user.target

==================================================================
GNU nano 5.4      /etc/systemd/system/frontend.service                
[Unit]
Description=Next.js Application (Frontend)
After=network.target

[Service]
Type=simple
WorkingDirectory=/home/youruser/frontend
ExecStart=/usr/bin/yarn run start
Restart=on-failure
User=youruser
Environment=NODE_ENV=production

# Adjust the limit as needed
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

================================================================== 
GNU nano 5.4        /etc/systemd/system/other-service.service                  
[Unit]
Description=Other Service
After=network.target

[Service]
Type=simple
WorkingDirectory=/home/youruser/other-service
ExecStart=/usr/bin/npm run start

Restart=on-failure

User=youruser

# Adjust the limit as needed
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

8) Increase VM Swap Memory

Because I use a single cheap VM to run everything, I need extra memory. But I don't want to increase the VM cost, so the workaround is to increase the VM swap memory. Below are the steps, but please be careful when choosing the hard disk size for your swap memory, because the change is permanent even after a restart

top
sudo swapon --show
free -h
df -h
top
sudo fallocate -l 3G /swapfile
ls -lh /swapfile
sudo chmod 600 /swapfile
ls -lh /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo swapon --show
free -h
sudo cp /etc/fstab /etc/fstab.bak
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
cat /proc/sys/vm/swappiness
cat /proc/sys/vm/vfs_cache_pressure
sudo sysctl vm.vfs_cache_pressure=50
top

9) Install & Set Up Firewall

In my case, if I don’t setup the firewall, every port in my machine is accessible with public IP of my VM. So that, to increase the security, I setup UFW to close access for all port except the https and SSH port

sudo apt install ufw
sudo ufw status
sudo ufw status verbose
sudo ufw show profiles
sudo ufw enable
sudo ufw allow ssh
sudo ufw default allow incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw enable
sudo ufw status verbose
sudo ufw deny 3000
sudo ufw deny 9002
sudo ss -tulnp
sudo ufw allow 80
sudo ufw allow 443
sudo ufw default deny incoming
sudo ufw status verbose
sudo ufw reload
sudo ufw status numbered
sudo ufw delete 2
sudo ufw delete 6
sudo ufw reload
exit

That's all about the simple architecture for deploying my personal website. Please note that this architecture is not recommended for commercial use, as the components and layers used here are still the cheapest, and might fail to handle High RPS traffic.