⚡ TL:DR

We will cover how to efficiently create a WebSockets API Gateway and send real-time messages from our Lambda functions to our WebSockets client to make our applications more interactive.

🌍 Introduction

In today’s fast-paced digital landscape, the demand for instant communication and real-time updates has become more and more important. Whether it’s receiving breaking news alerts, tracking deliveries, or staying connected with friends and colleagues, the need for timely information has never been greater. In such a dynamic environment, traditional methods of communication often fall short, unable to provide the immediacy and responsiveness required.

This is where real-time notification systems step in to bridge the gap. By leveraging technologies like WebSockets, these systems enable seamless, bidirectional communication between clients and servers, allowing for instantaneous updates and alerts.

In the past year, I’ve completed a project for a client that was using a fully event-driven architecture, and for me, it was missing a crucial feature, real-time notification.

💪 Goals and Objectives

In this article, we will dive into the creation of a simple system on AWS by providing an architecture diagram, a few principles, the limits of performing this architecture on AWS, and a little bit of code.

We will be using the following AWS services:

• AWS API Gateway (Rest API and WebSockets API)

• AWS Lambda

• AWS Simple Queue Service

• AWS DynamoDB

🛠️ Requirements

In this article, we will mostly be using AWS so make sure you have an account ready to play with.

🤔 Solution Architecting

We have a simple web application that performs Rest API calls, some methods will create new AWS EC2 instances and take some time, we want to be notified through the whole creation process, at every step.

Here we are, with a blank page, tons of ideas in mind but can’t get them out. We know what we are looking for, a simple notification system. While some AWS services out there perform notification such as SNS (Simple Notification Service), it might not be our best fit for real-time notification.

When thinking about web and Rest API, an idea would be to store notifications in a database and make queries to our API that fetch stored notifications. But how should we fetch these notifications? Should we add a button to our web application that fetches these notifications? Should we create code that automatically makes the request to the right API endpoint every X seconds and collects the notifications? This is a solution, but not the one we are looking for.

What we want to do is real-time, a protocol that comes into my mind is WebSockets. WebSocket is a communication protocol that provides full-duplex communication channels over a single, long-lived connection between a client and a server. Unlike traditional HTTP connections, which are stateless and involve a request-response cycle meaning that the connection is terminated when the response is received, WebSockets enable persistent, low-latency communication.

Diagram explaining simply how WebSockets works

With that definition in mind, WebSockets seems to be a solid candidate for our problem. On top of that, the AWS API Gateway service allows us to create WebSockets API, just what we need.

We now have to think about how we correlate our Rest API requests and WebSockets communication. If we take a look at how WebSockets API Gateway works on AWS, we might find one solution. If we refer to the AWS documentation, to send a message to a WebSocket client, we need to perform an HTTP request to the following URL:

POST https://{api-id}.execute-api.us-east-1.amazonaws.com/{stage}/@connections/{connection_id}

Here, the interesting part of the URL is “{connection_id}”. That means that each time a user connects to the WebSockets API Gateway a connection ID is generated. Don’t we have a similar feature on Rest API Gateway? When requesting Rest API Gateway, a Request ID is generated, awesome! We now have a way to correlate WebSocket Connection and Rest API requests. For each request made to the Rest API Gateway, we will have to provide the WS connection ID. We can use a database solution to store these IDs and use them in our system to send messages later.

We have to find a way to send messages from AWS to our web application. Since we are dealing with an application that creates instances, we may want to send messages once instance creation starts, when instances are successfully created, and when instances are usable. Wouldn’t it be convenient to have a centralized way of delivering messages?

First thing first, we saw earlier that sending a message to one WebSocket client can be done by making a POST HTTP request to a specific URL. We can create a Lambda Function that will perform this HTTP request. In order to invoke that lambda function, we can either invoke the Lambda Function directly from other lambda functions, we can also create an SQS Queue that will receive every message and then send them to our Lambda Function, this is the solution that we will use because it decouples our architecture, it is more resilient to failure and it gives us more visibility about the number of messages to process.

We have one last topic to address, we want to send messages when the instance is created and ready to be used, but how? We can leverage Amazon EventBridge and catch those events.

At the end, we should have the following architecture:

Simple diagram (using Excalidraw) of the solution described above using API Gateway (Rest API/WebSockets), Lambda functions, DynamoDB and a SQS Queue

🛠️ Implementing our solution

Before actually starting to build our solution, let’s give some context here and define some boundaries:

• In the provided architecture diagram above, we have mentioned the use of an SQS queue. We won’t be implementing it here.

• We’ve also suggested the use of DynamoDB to store Requests IDs and remove them on disconnection, we won’t be implementing them on here.

• We won’t deploy a Rest API Gateway as well, we will simply deploy a WebSockets API Gateway and create a Lambda that we will trigger manually to send back messages to our WebSocket client.

We have a simple web application that allows users to create VDI instances (Virtual Desktop Interface) on the go. The application is fairly simple, users are authenticated and can request their instance from the web interface.

Users would like to get in real-time notifications about their order of instance.

Very simple web application design that demonstrate our use case

Here, users want to create new instances. Creation of EC2 instances can take some time due to many factors:

• EC2 creation time

• Instance setup (using any automation solution)

• Compliance check

The user requested to get an update at each step of the instance creation process. Let’s create our notification system, starting on AWS!

First, let’s log in to AWS and head to the API Gateway service. Create a new WebSocket API Gateway.

When communicating to a WebSocket API Gateway, you send a payload, in this payload you will need to specify the action you are trying to perform. In this input, you set which key inside of your payload is the action you are trying to perform. We will come back to this later.

Add the $connect and “$disconnect” predefined routes to your API Gateway.

• $connect: This route is the one called when your client connects to the WebSocket API Gateway

• $disconnect: This route is the one called when your client disconnects to the WebSocket API Gateway

In our use-case, we don’t need to add any custom routes yet.

Then you will want to create some Lambda Functions, one for the $connect route, and another one for the $disconnect route. This is the Lambda function that will be called when the client connect/disconnect to our API Gateway. Remember earlier when we said that we will have to match WebSocket connection ID and Rest API Request ID? Well, the $connect lambda function is a good place to write our logic that store the connection ID somewhere.

In the $disconnect lambda, we can write the logic that removes our stored connection ID.

We will write the lambda code later.

The final step is the stage. Here we are testing things, so we will simply create a first stage named “development”. Click on “Next” and deploy your API Gateway.

Once our WebSocket API Gateway is deployed, let’s try it. We will use Postman which supports WebSockets.

Grab your API Gateway WebSocket URL and open Postman.

Click on Connect, and you shall see that the connection is successful. Great!

We will add some code to our $connect Lambda function to analyze what’s being passed to that lambda.

import json

def lambda_handler(event, context):

    print(json.dumps(event))

    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

This code will simply dump every passed parameter to that Lambda. Let’s disconnect and then reconnect to our WebSocket API and see those parameters.

{
    "headers": {
        "Host": "c9if5l9rgd.execute-api.eu-west-3.amazonaws.com",
        "Sec-WebSocket-Extensions": "permessage-deflate; client_max_window_bits",
        "Sec-WebSocket-Key": "q2ueME10A76weGCzBqgq2Q==",
        "Sec-WebSocket-Version": "13",
        "X-Amzn-Trace-Id": "Root=1-65feee73-002d1ca83f839f353a4c89b4",
        "X-Forwarded-For": "xxx",
        "X-Forwarded-Port": "443",
        "X-Forwarded-Proto": "https"
    },
    "multiValueHeaders": {
        ...
    },
    "requestContext": {
        "routeKey": "$connect",
        "eventType": "CONNECT",
        "extendedRequestId": "VFoyGG1vDoEF2dg=",
        "requestTime": "23/Mar/2024:15:00:03 +0000",
        "messageDirection": "IN",
        "stage": "development",
        "connectedAt": 1711206003575,
        "requestTimeEpoch": 1711206003576,
        "identity": {
            "sourceIp": "xxx"
        },
        "requestId": "VFoyGG1vDoEF2dg=",
        "domainName": "c9if5l9rgd.execute-api.eu-west-3.amazonaws.com",
        "connectionId": "VFoyGcaIjoECIjw=",
        "apiId": "c9if5l9rgd"
    },
    "isBase64Encoded": false
}

Did you notice the important parameter here? “connectionId” that has a value of “VFoyGcaIjoECIjw=”. Note that this number is unique and generated on the client connection.

We will now create a new Lambda Function that will send a message to a given WebSocket connectionId, to ensure that we are able to reach this connectionId.

import json
import boto3

api_gw_id = "c9if5l9rgd" # Change with your WebSocket API Gateway ID
region = "eu-west-3" # Change with the region where your WebSocket API Gateway ID is

def send_message(connectionId: str, message: str, stage: str) -> None:
    client = boto3.client(
        "apigatewaymanagementapi",
        endpoint_url=f"https://{api_gw_id}.execute-api.{region}.amazonaws.com/{stage}",
    )
    client.post_to_connection(
        ConnectionId=connectionId, Data=json.dumps(message).encode("utf-8")
    )

def lambda_handler(event, context):  
    send_message(
        connectionId=event["connectionId"],
        message=event["message"],
        stage=event["stage"]
    )
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

Create a new test, and set these values:

{
  "connectionId": "VFoyGcaIjoECIjw=", # The Connection ID caught in the $connect dump
  "stage": "development",
  "message": "Hello Medium" # The message you want to send
}

We are not done yet! We need to give additional permission to our Lambda Function otherwise we will face a permission denial while invoking our test!

{
    "Sid": "Statement1",
    "Effect": "Allow",
    "Action": [
        "execute-api:ManageConnections"
    ],
    "Resource": [
        "arn:aws:execute-api:YOUR_REGION:YOUR_ACCOUNT_ID:API_GATEWAY_ID/*/*"
    ]
}

Now, execute your test. You should now see your message on Postman!

We’re not quite done yet. There is another thing to do, include theses messages in our web application.

There are tons of ways to perform WebSocket. Here we will use Vanilla JavaScript.

We will simply add the following code to our web application:

<script>
const socket = new WebSocket('ws://your-websocket-server-address');

// Event listener for when the connection is established
socket.addEventListener('open', function (event) {
    console.log('WebSocket connected');
});

// Event listener for incoming messages
socket.addEventListener('message', function (event) {
    // Display an alert with the received message
    alert('Message received: ' + event.data);
});

// Function to send a message to the WebSocket server
function sendMessage(message) {
    if (socket.readyState === WebSocket.OPEN) {
        socket.send(message);
        console.log('Message sent: ' + message);
    } else {
        console.error('WebSocket connection is not open.');
    }
}

</script>

Once this code is added, we can refresh the page. Our web application will then send a connect request to our WebSocket API Gateway. On the AWS side, just like before, we will fetch the connection ID, and manually run our Lambda function.

Screenshot of the web application with the message sent from AWS

And guess what? Our Web Application displays our message! We have made it!

👀 Room for improvements

Well, as you are probably wondering, I have mentioned earlier the use of a REST API Gateway, an SQS queue but not above, why?

The general idea was to provide you with a simple use case of how to use WebSockets within the AWS environment.

On our Web Application, we have a small interface that allows users to create virtual desktop instances. When clicking on the “Request New Instance” button, an HTTP request is sent to our REST API Gateway. When sending this request, we will need to send the Connection ID into our initial payload, that way we will be able to send notifications later, during our instance initialization process.

Each Lambda Function has to send a message to our SQS queue, to ensure message ordering and delivery. We can then either configure a Lambda that’s triggered on message pushed that will send messages to our WebSocket client or having a scheduler that will invoke our Lambda function and fetch messages from our SQS Queue.

🏁 Conclusion

We have learned how to use WebSockets on AWS, and it wasn’t that hard, right? I hope that you have enough resources to dig on your own, and maybe implement some of my ideas in one of your projects! Feel free to comment on this post with a link to your project, I will happily take a look at it.

We will cover how I have created my own Minecraft Server Provider using Discord to create the servers and ECS to run the servers.

🌍 Introduction

Since my teenage years, I have always been a fan of hosting my very own game servers. I used to have a well-known French community (WayCaster.fr) on a game named Garry’s Mod. We had a few servers that were up and full most of the time. I was using some open-source game panels, but none of them were interesting to me. So, I ended up using the classic Linux shell to manage the servers. Then I learned programming. I started with web development using PHP, and I thought, “Hey, let’s create my own game server panel!” It never happened. 🙄

Today, I have realized that I no longer use web apps when an alternative exists, such as phone applications or bots (Telegram, WhatsApp, Discord, etc.). So here I am now, a fully grown man with knowledge in coding, solution architecture, and the same passion for building stuff.

I would like to build a solution that allows Minecraft Players from the Mineral Contest community to start a server on the go, using Discord.

What is Mineral Contest? Mineral Contest is a Minecraft game mode that went viral in France few years ago. A French Youtuber name “Squeezie” released a YouTube video of him and his friends playing this game mode and people loved it. (for the curious ones: https://www.youtube.com/watch?v=q_xALLkdwNA)

In a few words; the main objective is to collect resources (Iron, Gold, Diamond, Emerald) and bring them to your base to earn some points. You are playing against 3 other teams, each team is usually composed of 4 players.

This is where I came in! The game mode was private, and everyone wanted to play it. So I decided to re-create it based on the YouTube Videos and release it publicly. The plugin quickly became popular, and is still active today!

2024/08/08 — Current statistics for the plugin — available at https://mc.monvoisin-kevin.fr/

💪 Goals and Objectives

In this article, we will dive into the creation of a system that provides Minecraft Servers using AWS, Discord, and a NodeJS bot.

We will be using the following AWS services:

• AWS ECR (Elastic Container Registry)

• AWS ECS (Elastic Container Service)

• AWS DynamoDB

We will also be using the discord.js library to build our bot and the AWS SDK for JavaScript.

🛠️ Requirements

You will need an AWS Account, a Discord Application, Docker, NodeJS (version used in this article: 20.15.1).

The source code will also be provided, so you might need to use Terraform, Git, and the AWS CLI.

🤔 Solution Architecting

First thing first, what are we looking for? What are the requirements? What is the context?

Alright, so we are looking for a solution using Discord as our application client, we will be using AWS to host any virtualization service and database. We will also need to write a discord bot. Many choices to make here!

Firstly, for the Discord Bot, I would like to use NodeJS to practice this language. Lucky me, there is a well-known Discord library for Javascript named Discord.js. I want to restrict server creation to a specific set of users, to achieve this behavior, I will need to create a Discord Server Role.

How does the user should request its server? Should it be via a message sent to the moderation team? Via a server command? Here, I want the service to be fully automated, using a server command to provide automation.

Now that the user can request a server using a command, how does the user get his server details? I am not a fan of private messages, so this is not a suitable option. However, in my Discord Server, I am using a Ticketing tool. Users click on a button, it creates a new text channel that only the user and administration team can see; I like this idea a lot. Every user that requests a game server will have access to a newly created text channel that will contain every detail of its game server. Great!

Now, I need to think about server hosting, how am I going to create the game servers? We have many options here:

1. EC2 instance: Either create a Golden AMI (an Amazon Machine Image that contains any dependencies and server binaries, pre-built that can be used without any further configuration) or use a configuration solution like AWS System Manager (Automation or RunCommand).

2. ECS Cluster: Using ECS Fargate we don’t have to manage any EC2, we only have to create our Docker Image, and specify the CPU and Memory needed.

3. EKS Cluster: Using Kubernetes, we don’t have to manage any EC2, we only have to create our Docker Image, and specify the CPU and Memory needed, however, this solution sounds a bit overkill for our usage and is expensive.

On a side note, I do not need to have a persistent EBS volume since the game server will be temporary.

We now have to choose between an EC2 instance and an ECS Cluster. I already have experience using EC2 instances, coupled with SSM and I would like to learn and use this project to get knowledge about ECS so I will be selecting ECS.

We have to figure out how to create a game server on ECS. We have many possibilities here:

• Using a service: a service is a feature on ECS that guarantees a specific amount of containers up and running. Kinda like the AutoScaling Group on EC2. If a task fails or gets stopped, the service will automatically shut down the task and start a new one.

• Using a task: a task contains a container to use, CPU and Memory to allocate, and Network configuration. This option is lighter than using a service and more suitable for what we are looking to achieve.

Let’s recap what we’ve said so far:

1. The user needs to have a special role

2. The user needs to use a Discord command to request a server creation

3. A new ECS task has to be created

4. The details of the ECS task need to be retrieved (IP Address, Current State, …)

5. A new Discord text channel needs to be created, only the administration team and the server owner can have access to it.

6. Send the details to the newly created text channel.

We now have to think about data persistence. How do we ensure the bot has a list of active ECS tasks, knows when the task was created, how long before the tasks need to be killed, how to ensure the bot keeps this data stored somewhere “safe” (i.e: not in memory) in case the bot crash and has to restart?

We have here many solutions:

1. Use a memory database (Redis, H2, …)

2. Use a SQL database (MariaDB, MySQL, SQL Server, …)

3. Use a No-SQL Database (MongoDB, DynamoDB, …)

Since this is my own money, I want the solution to be cheap, nearly free. DynamoDB would be a solid choice since it’s managed, almost free for my usage, and runs outside of my bot application.

Let’s create a diagram showing what our solution would look like:

Currently, I am happy with the choices we made, let’s begin implementing our solution!

🛠️ Implementing our solution

Before actually implementing our solution, I just want to give some words. I won’t be going through every step required to build this project. I will go through basic things, and the whole project will be available at the end of this post. Thank you if you already made it that far!

Our first step here is to create our discord application. Simply head to https://discord.com/developers/applications and click on “New Application”. Generate a new Token, grab the Client ID, and save it somewhere, we will need it later.

We will now create the bot application logic. Based on the Discord.js documentation, installing the library is as easy as 1–2–3. Simply run the following command:

npm install discord.js dotenv

And now, create our file first:

// main.js
import { Client, GatewayIntentBits } from 'discord.js';
const client = new Client({ intents: [GatewayIntentBits.Guilds] });

client.on('ready', () => {
  console.log(`Logged in as ${client.user.tag}!`);
});

client.login("token");

Run the bot with the following command:

$ node main.js                                                                                                                                      N
Logged in as bot1271486725746720900#4545!

Awesome! We now have the basic code that allows our bot to log in! For those who want to explore a little bit more, the “Getting Started” page of Discord.Js is full of useful information, you can read more here.

Let’s now create our application structure, with files and folder names. Here, I will be using the following structure:

medium-discord-bot-aws/
├── commands/
│   └── typeOfCommand/
│       ├── commandExecutor1.js
│       └── ...
├── events/
│   ├── onMessage.js
│   ├── onReady.js
│   └── ...
├── .env
├── deploy-commands.js
├── main.js
└── package.json

• In the commands folder, we will have every command that our bot will support.

• In the events folder, we will define every event that our bot will listen to.

• In the .env file, we will have every environment variable required and read by our bot.

• In the deploy-command.js file, we will have the logic to register commands into our discord server.

• In the main.js file, we will have our application code.

• The package.json file will contain the project description, dependencies, the main script to execute, and a little bit more.

With that structure in mind, let’s add and create the deploy-commands.js file.

const { REST, Routes } = require('discord.js');
const fs = require('node:fs');
const path = require('node:path');
require('dotenv').config()


const commands = [];
// Grab all the command folders from the commands directory you created earlier
const foldersPath = path.join(__dirname, 'commands');
const commandFolders = fs.readdirSync(foldersPath);

for (const folder of commandFolders) {
 // Grab all the command files from the commands directory you created earlier
 const commandsPath = path.join(foldersPath, folder);
 const commandFiles = fs.readdirSync(commandsPath).filter(file => file.endsWith('.js'));
 // Grab the SlashCommandBuilder#toJSON() output of each command's data for deployment
 for (const file of commandFiles) {
  const filePath = path.join(commandsPath, file);
  const command = require(filePath);
  if ('data' in command && 'execute' in command) {
   commands.push(command.data.toJSON());
  } else {
   console.log(`[WARNING] The command at ${filePath} is missing a required "data" or "execute" property.`);
  }
 }
}

// Construct and prepare an instance of the REST module
const rest = new REST().setToken(process.env.DISCORD_TOKEN);

// and deploy your commands!
(async () => {
 try {
  console.log(`Started refreshing ${commands.length} application (/) commands.`);

  // The put method is used to fully refresh all commands in the guild with the current set
  const data = await rest.put(
   Routes.applicationGuildCommands(process.env.DISCORD_CLIENT_ID, process.env.DISCORD_GUILD_ID),
   { body: commands },
  );

  console.log(`Successfully reloaded ${data.length} application (/) commands.`);
 } catch (error) {
  // And of course, make sure you catch and log any errors!
  console.error(error);
 }
})();

When you execute this script, the bot will automatically register any command location in our “commands” folder to our discord server. Great! For those wondering, I did not write all this by myself 👀, the code came from the Discord.js documentation available here.

We will now edit our main.js file to include command processing event processing, and auto-completion on our commands.

const fs = require('node:fs');
const path = require('node:path');
const { Client, Collection, GatewayIntentBits, Events } = require('discord.js');

require('dotenv').config()

const client = new Client({ intents: [GatewayIntentBits.Guilds, GatewayIntentBits.GuildMessages, GatewayIntentBits.MessageContent] });

client.commands = new Collection();
const foldersPath = path.join(__dirname, 'commands');
const commandFolders = fs.readdirSync(foldersPath);

for (const folder of commandFolders) {
 const commandsPath = path.join(foldersPath, folder);
 const commandFiles = fs.readdirSync(commandsPath).filter(file => file.endsWith('.js'));
 for (const file of commandFiles) {
  const filePath = path.join(commandsPath, file);
  const command = require(filePath);
  if ('data' in command && 'execute' in command) {
   client.commands.set(command.data.name, command);
  } else {
   console.log(`[WARNING] The command at ${filePath} is missing a required "data" or "execute" property.`);
  }
 }
}

const eventsPath = path.join(__dirname, 'events');
const eventFiles = fs.readdirSync(eventsPath).filter(file => file.endsWith('.js'));

for (const file of eventFiles) {
 const filePath = path.join(eventsPath, file);
 const event = require(filePath);
 if (event.once) {
  client.once(event.name, (...args) => event.execute(...args));
 } else {
  client.on(event.name, (...args) => event.execute(...args));
 }
}

client.on(Events.InteractionCreate, async interaction => {
 if (interaction.isAutocomplete()) {
  const command = interaction.client.commands.get(interaction.commandName);

  if (!command) {
   console.error(`No command matching ${interaction.commandName} was found.`);
   return;
  }

  try {
   await command.autocomplete(interaction);
  } catch (error) {
   console.error(error);
  }
 }
});



client.login(process.env.DISCORD_TOKEN);

Don’t worry, the whole code will be available at the end of this blog post. Now that we have the basics of the bot, let’s head into the AWS part.

To provision any resource, we will be using Terraform.

“Terraform is an infrastructure as code tool that lets you build, change, and version cloud and on-prem resources safely and efficiently.” — https://developer.hashicorp.com/terraform/intro

To make things right, we will start by creating an S3 Bucket, and a DynamoDB table to store our remote state. That way, the infrastructure state isn’t saved on our computer and we cannot remove it accidentally. Enable Object Versioning on your S3 bucket to ensure the terraform state file cannot be deleted accidentally and can be retrieved if any accident happens. While we are here, also create an ECR repository.

Here is our provider.tf file:

// provider.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket = "mybucket"
    key    = "path/to/my/key"
    region = "eu-west-1"
  }
}

provider "aws" {
  region = local.region
}

I like to work with configuration files rather than terraform variables. This is my own opinion. Here I will be working with a YAML configuration file and locals.

# config.yaml
region: eu-west-1
cluster_name: MineralContest
ecr_repository: your_repository_id.dkr.ecr.eu-west-1.amazonaws.com
image_name: mineralcontest

// locals.tf
locals {
  raw_data         = yamldecode(file("${path.module}/config.yaml"))
  region           = local.raw_data["region"]
  ecs_cluster_name = local.raw_data["cluster_name"]
  ecr_repository   = local.raw_data["ecr_repository"]
  image_name       = local.raw_data["image_name"]
}

Our first terraform resource will be the VPC. I will be using the AWS VPC Module. Here is the vpc.tf file.

// vpc.tf
module "vpc" {
  source = "terraform-aws-modules/vpc/aws"
  name = "mineralcontest-vpc"
  cidr = "10.0.0.0/16"
  azs            = ["${local.region}a"]
  public_subnets = ["10.0.1.0/24"]
}

Then, we will create our ECS cluster, the ECS task definition, and the security group that our task will use. Here is the ecs.tf file

// ecs.tf 
resource "aws_ecs_cluster" "cluster" {
  name = local.ecs_cluster_name
}

resource "aws_ecr_repository" "repository" {
  name = local.ecr_repository
  image_tag_mutability = "MUTABLE"
  image_scanning_configuration {
    scan_on_push = false # Disable image scanning
  }
}

resource "aws_ecs_task_definition" "mineralcontest_task" {
  family                   = "mineralcontest"
  task_role_arn            = aws_iam_role.ecs_task_role.arn
  execution_role_arn       = aws_iam_role.ecs_task_role.arn
  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  cpu                      = "2048"
  memory                   = "4096"
  container_definitions = jsonencode(
    [
      {
        name      = "mineralcontest",
        image     = "${local.ecr_repository}/${local.image_name}:latest",
        cpu       = 0,
        memory    = 128,
        essential = true,
        portMappings = [
          {
            containerPort = 25565,
            hostPort      = 25565,
          },
        ]
      },
    ]
  )

}

resource "aws_security_group" "mineralcontest" {
  vpc_id = module.vpc.vpc_id
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 25565
    to_port     = 25565
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 25565
    to_port     = 25565
    protocol    = "udp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Now, let’s create our iam.tf file that will contain every needed policies and roles.

// iam.tf
resource "aws_iam_instance_profile" "ec2_ssm_instance_profile" {
  name = "EC2-SSM-Instance-Profile"
  role = aws_iam_role.ec2_ssm_instance_role.name
}

resource "aws_iam_role" "ec2_ssm_instance_role" {
  name               = "EC2-SSM-Instance-role"
  assume_role_policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "ec2.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
EOF
}


resource "aws_iam_role" "ecs_task_role" {
  name               = "ECS-Task-role"
  assume_role_policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "ecs-tasks.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
EOF
}

resource "aws_iam_policy_attachment" "ecs_attachment" {
  name       = "ECS-Task-attachment"
  roles      = [aws_iam_role.ecs_task_role.name]
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

We’ve also mentioned the use of a DynamoDB table to store our current servers up and running, so let’s create the table.

// dynamodb.tf
resource "aws_dynamodb_table" "tasks" {
    name           = "mineralcontest_tasks"
    billing_mode   = "PAY_PER_REQUEST"
    hash_key       = "taskId"
    attribute {
        name = "taskId"
        type = "S"
    }
}

We are done with the AWS part. Now, we need to create our game server docker container image. In my use case, I want users to be able to become administrator by themself. So I have written a simple Java plugin that allows an user to become an admin by submitting a game command. I have also created a small script that is the entry point of the container and pass the token to the game server (for the user to become admin), plus the configuration required to make the server “official” or “non-official” (enabling or not non-official game version). Here is my dockerfile.

# Build the Spigot Server
FROM amazoncorretto:17-alpine3.16 as SPIGOT_BUILDER
WORKDIR /spigot
ARG version=1.19.4
RUN apk update && apk add curl git
RUN curl -o BuildTools.jar https://hub.spigotmc.org/jenkins/job/BuildTools/lastSuccessfulBuild/artifact/target/BuildTools.jar
RUN java -jar BuildTools.jar --rev $version --final-name server.jar


# Build the MineralContest Plugin
FROM maven:3.9.8-amazoncorretto-17-al2023 as MINERALCONTEST_BUILDER
WORKDIR /mineralcontest
RUN yum install -y git
RUN cd /mineralcontest && git clone https://github.com/synchroneyes/mineralcontest
RUN cd /mineralcontest/mineralcontest && mvn clean install

# Create the final image
FROM amazoncorretto:17-alpine3.16 as MC_SERVER
WORKDIR /server
COPY --from=SPIGOT_BUILDER /spigot/server.jar server.jar
COPY MinecraftPlugins/OPRedeem.jar plugins/OPRedeem.jar
COPY --from=MINERALCONTEST_BUILDER /mineralcontest/mineralcontest/target/MineralContest.jar plugins/MineralContest.jar
COPY files/server.properties server.properties
COPY scripts/init.sh init.sh


EXPOSE 25565
CMD ["/server/init.sh"]

We now have to build, tag, and push our image to the ECR repository. The ECR service gives us the command required to build, tag, and push. Let’s execute them.

aws ecr get-login-password --region eu-west-1 | docker login --username AWS --password-stdin your_repository_id.dkr.ecr.eu-west-1.amazonaws.com
docker build -t mineralcontest .
docker tag mineralcontest:latest your_repository_id.dkr.ecr.eu-west-1.amazonaws.com/mineralcontest:latest
docker push your_repository_id.dkr.ecr.eu-west-1.amazonaws.com/mineralcontest:latest

And voila! Our image is now pushed into our ECR repository and is ready to be used!

The (almost) final steps in our project are to create the command handler for the server creation, server stop, and automatic server shutdown after a specific amount of time. I won’t be covering every command otherwise this post will be much longer (it’s already really long, sorry about that 🙄). The whole code will be available at this end, so feel free to dig and take what’s interesting to you.

Finally, we need to package our bot application. I want this project to be as cheap as possible, for the bot hosting, I will be using a server that I am already renting and has Docker installed on it. Here is the dockerfile and the docker-compose.yaml file for the bot application

FROM node:20.15-alpine
WORKDIR /app
COPY . .
RUN npm install
RUN npm run deploy-commands
CMD ["npm", "start"]

// docker-compose.yaml
version: '3'
services:
  mineralcontest_bot:
    build:
      context: .
      dockerfile: Dockerfile
    restart: always
    network_mode: host

Now, simply run docker-compose up -d and the bot is now running! Awesome, we did it, and everything is working smoothly. Let’s talk about cost optimization.

As mentioned earlier, this game server is running the game mode Mineral Contest, which last approximately 60 minutes. It can be more or less depending on the user’s settings. We can implement a feature that automatically terminates an ECS task after a specified amount of seconds. Here is the code I wrote for this feature:

const { getLocalDatabase } = require('../services/serverDatabase');
const { killServer } = require('../services/killServer');

require('dotenv').config()


/**
 * Watch servers and kill them when maximum run time is reached
 * @returns Interval handler
 */
function watchServerRunTime() {
  return setInterval(async () => {
      const localDatabase = getLocalDatabase();
      for(let taskId in localDatabase){
          let task = localDatabase[taskId];

          let creationDate = task.creationDate;
          let currentDate = Date.now();
          if(currentDate > creationDate + process.env.GAMESERVER_MAX_DURATION){
              await killServer(taskId);
          }

      }
  }, 1000);
}

module.exports = { watchServerRunTime };

Every second, the bot will fetch the local database (a copy of the DynamoDB table stored locally, to avoid excessive API calls) and check when the server was created. If the server was created more than 90 minutes ago, then the ECS task is killed.

I want to implement another feature. I want to ensure the server has at least 4 players in it. If this requirement is not met, then the server is terminated. To implement this feature, I have used a JavaScript library that allows Minecraft server queries to fetch the current player count. When the minimum player count threshold is not met, a warning is sent to the user’s Discord text channel. After 3 warnings, the server is automatically killed.

👀 Room for improvements

That was a long post, wasn’t it? A first improvement would be to cut this post into subsidiary posts and add a “previous” or “next” link. Let me know your thoughts about this idea!

About this project, the first improvement would be within the game server container. Currently, I am using environment variables. I could improve it by using Secrets which is supported using Docker Compose which is a feature I am currently using. https://docs.docker.com/compose/use-secrets/. I would have to make some changes to my application and read secrets from /run/secrets/<secret_name>.

For the build of the game server container, I could use a CI/CD to automate the build and deployment. Since I am cloning a Git Repository, the version my container has of this repository could be outdated. I can use CodeBuild to build my docker images and push them to my ECR repository and EventBridge Scheduler to schedule the build.

As you can see in the diagram above, I have added the SNS Service. Currently; we lack some observability in our application. We could add some monitoring, for instance when a server creation fails, when a new Docker image build fails or succeeds, and when anything goes wrong, a notification can save us a lot of time.

Also, within the CI/CD, it would be nice to have some sort of testing, code scanning, and convention. We could do the following:

1. Ensure the code is formatted everywhere and use the same naming convention (snakecase/camelcase/…)

2. Ensure there are no over-permissive policies that could lead to big issues

3. Local deployment to ensure everything is ready to be used

Currently, the application only supports one type of game server. It would be a nice idea to add a way to support multiple games. What’s possible is to define some sort of game server template. You would define your game server settings in a JSON file, and upload the file into the Discord Server. The Bot application would parse the file, store the details in a DynamoDB table, and make the game server available for creation.

The bot application now! It was written using SDKv2 instead of SDKv3. AWS will end support for SDKv2 in 2025, meaning this project would need some parts to be rewritten in 2025.

🏁 Conclusion & Source Code

This was a really fun project where I have learned a lot. I had no knowledge of ECS and very poor coding skills in Javascript. I actually enjoyed this project a lot.

The application is now live within my community, and some servers have already been provided to players, they are happy, and I am happy, what else?

Thank you for reading this whole article, I hope that wasn’t too boring to read, the source code is available below, feel free to read the README.md file to understand how to deploy the solution.

Source code: https://github.com/Synchroneyes/gameserver-discord-aws-mc

I am excited to read your remarks!