Shotbow

This is going to be a slightly more technical read than most of our postings, you have been warned.

After a hiccup or two, we have completed an (almost) invisible fundamental change to our server infrastructure: Every one of our Minecraft servers is now deployed into a Kubernetes cluster for orchestration, instead of a home grown orchestration layer we’ve been using for over 5 years we’ve simply referred to as “Deployment”.

Deployment: A Legacy



Deployment was an amalgamation of Python scripts, cron jobs, and database polling to keep up with the state of the servers, roll new ones, destroy expired ones, and give us insight into what was currently running.

When this first was created it solved a HUGE need and was miles ahead of our old method of “Login and run script A if we need another MineZ server, edit the server.properties, and execute the start command. Login and run script B if we need another Anni server, edit the server.properties etc...”.

It helped us break the model of server instances being uniquely managed (Anyone remember the old days of MINEZ_07_US loyalty?), and instead allowed us to have repeatable units we could scale out and in at will.



Over the years, developers have added and removed from it to automate additional common tasks and it has been very useful, but its deep underbelly became a scary place that no one dared to venture into, since it was impossible to untangle the web of scripts and schedules at its heart.

Sometimes rsyncs of the server files would fail and we’d have to dig through our servers to find what went wrong. Other times, some bad data broke one of its unspoken “rules” and it refused to work entirely. It became a gigantic, spooky, mysterious volcano that could erupt at any moment.

Ultimately the key shortcoming of Deployment was that it was tailor made to only run our Minecraft instances, what happens if we want to run something else like a new database, web service, or even host another game? We knew there had to be a better way, and we happen to have the background necessary to put it to work.

Enter Stage Right

Adam (aet2505) and I both work professionally as Software Engineers dealing with Kubernetes (k8s) on a daily basis.

I asked him what he thought it would take us to get everything running in k8s and we both sort of verbally shrugged and got to work. As many of you know, I am based in Salt Lake City, Utah, what you might not know is that Adam is not.

The time difference between us basically meant at least one of us could be working on it around the clock for almost 2 weeks straight, provisioning the cluster, establishing deployments, creating a dockerization strategy for our instances, and testing out how Bungee liked the networking model inside the cluster (spoiler alert: It loved it).

Every Rose Has Its Thorns

The biggest hurdle we hit along the way was that Kubernetes pods are inherently ephemeral, meaning they come and go at will, however we want our Minecraft servers to only shut down under specific circumstances.

To solve this, Adam established a Custom Resource Definition he aptly named “MinecraftSet” and created an operator that facilitates our desired control loop. This operator will keep the number of instances above the minReplicas, and a scalar can react to server state changes (like population) as well as ask if it is ready to shut down.

This lets game creators respond to shut down requests in the existing event driven pattern inside of Spigot and indicate whether it is safe or not. This has afforded us the flexibility to deploy a server that is usually fine to shut down on demand such as DBV and a server that should NEVER shut down in most circumstances like Annihilation using the same pattern.

Annihilation itself is able to own the logic governing shutdowns by influencing the control loop based on game state instead of a script running on a mystery box (could you imagine every game terminating just as Phase 3 starts!?).

Like Popsicles In Summer

Once we overcame that hurdle, we found a few problems that existed in our deployment world had simply melted away.

Because of Kubernetes’ networking model, all Minecraft servers could simply run on port 25565. Every instance is deployed into a “pod” in the cluster, and receives its own IP address. We had swathes of code for figuring out which port was available and ensuring that port is published when the server registered with our router that was simply deleted (deleting code is the BEST).

Managing storage was a bit of a pain, but PersistentVolumes in Kubernetes and volume mounts in Docker allow us to to handle storage in a way that is transparent to the server running. Using Prometheus, we are able to discover and scrape metrics on running instances automatically, allowing us to create useful Grafana dashboards for everything from server health to player report spikes.

ArgoCD helps us reconcile our repository of Kubernetes manifests with the running cluster, provide single button rollback support, and a quick glance view into what is running as well as server logs. Previously reverts consisted of tracking down the plugin, configuration, or build that had changed, manually reverting it, then re-releasing.

In this universe we can rollback with ArgoCD, which takes the pressure off the team so they can take their time diagnosing what went wrong with a rollout and easily roll forward again with the fix. Here’s When in Rogue in the ArgoCD interface:


One of the new additions I am working on is a proper social system, starting with Friends and soon with network-wide parties. This has a backing web API and is built on a graph database which is well suited for this kind of problem.

I was able to spin up the dev stack in minutes, with full discoverability and connectivity from our Minecraft instances to interact with it. This has allowed me to move quickly to create rich social features that will improve your experience with us here at Shotbow. This has also enabled some of our new devs to dive right into crafting new experiences, ensuring a safety net so those working with established modes can experiment and rollback easily, and has also increased our ability to monitor and support the infrastructure that powers these games we love.


If you enjoyed this technical deep dive into Shotbow's core infrastructure, be sure to let us know! We may just continue to do more of these "Devblogs" in the future!

But for now, as always,

Thanks for flying Shotbow!

Hello fellow SMASH-ers!

Today I bring you all a very special announcement regarding something we have been working on behind the scenes for quite a while now, and both I and the team are very excited to announce the first official SMASH tournament!

Tournament Registration
For this tournament, we will be using a third party site to conduct registration, brackets, and match results. We have a nice, short link for everyone to get registered as quickly as possible; smash.gg/shotbow. Smash.gg has lots of tools which allow us to neatly represent rules and regulations, who the Admins are, and how they can be contacted for assistance if necessary. The brackets and seeding can easily be done through the site, which will make match-ups as smooth as possible for players.

Below is a short video which will show you how registration through smash.gg will look, along with a step-by-step guide.

• Go to smash.gg/shotbow
• Click on either of the registration buttons
• Enter your "gamertag" which must be your in-game-name
• Enter your preferred email, password, birthdate, and country (we will not be using these other than for your initial "account registered" email) I recommend putting your name as your IGN if possible, since your name shows up on the attendees list and is thus shown to others, as you see me do in the video.
• Below it is also recommended to put your timezone, which you can do to ensure you are matched with a player from your timezone in the first match, and so we can compromise on schedules for further matches.
• Lastly, solve the captcha and check off the two boxes. From there you are all ready to go for your matches! Any information displayed about times or what bracket you are in is not set in stone, and will change by October 1st
• Youtube Video Link

Important Dates
Registration opens up today and will close on September 28th, 2020. Because we are unsure of how many players will choose to sign up for this event, the amount of pools and bracket settings must be modified after everybody has registered. We will be taking 2-3 days to do this behind the scenes, and the tournament will officially begin with all brackets and 1st round game times visible to the public by October 1st. However, this doesn't mean matches will begin on that date. We are hoping to conduct most matches in the evenings on Fridays-Saturdays throughout October so we don't conflict with players' or Admins' schedules. We are hoping to finish the tournament by the end of October, however this may change depending on how many players register.
Matches will be run in a 1v1 format all the way through the tournament, however do note if there is an odd amount of players we may have to accommodate in some way.

Tournament Rules + Setup
Smash.gg has all of the rules laid out on the homepage in a neat visual format, however I'll go over how everything will run here.

An Admin must be present at the time which your match is set to start, if an Admin is not present and you have not been given prior notice via discord, ingame/forum messages, or by email, feel free to ping myself or one of the other registered Admins listed on the homepage on discord in the #smash chat. The game will be carried out in a party match, with slightly modified itemsets. The Fishing Rod and the Extra Life items will be removed for this tournament, the timer is the standard 10 minutes, and lives is set to 3. Map selection will be done randomly by the host, although a few maps have been removed from our selection to ensure fair and competitive play. Once the match begins, the Admin will kill themselves 3 times via a command, it is important to not hit them during this time as that can influence how the final scoreboard will be. The winner will be whoever is left standing after the time is up, or in the case of both players having a life remaining, the scoreboard will decide who is the winner. Winners progress on to the next match at a scheduled time via smash.gg, and losers, well, lose.
There is no best 2 out of 3 format, nor is there a losers bracket, it is simply a single 1v1 match to decide a winner (however we may look into doing things like this at a later date for further tournaments). Matches will also be recorded, and the final outcome through the scoreboard will be posted in the discord for everybody to see, and the smash.gg site will update shortly after the match concludes to show the winner.

Hacked clients and such are forbidden, and any form of cheating or bug abuse will result in a disqualification. Failure to show up within 10 minutes of your assigned time will also result in an automatic forfeit, and if both players fail to show up, we will do a coin flip to decide a winner. If a player disconnects or lags out during a match, we will decide how to proceed on a case-by-case basis.

Prizes
・1st Place: 1 Month Shotbow Subscription + 50,000 Shotbow XP
・2nd Place: 40,000 Shotbow XP

Please be sure to only register once, if you accidentally register multiple times or lose access to your account, please contact a Smash staff member through the discord or as a reply to this post.

A Note from the team
This, to my knowledge, is the first event we've hosted like this before officially. There have been player hosted tournaments in the past, and we hope to live up to expectations of what an official tournament can be for the gamemode. We sincerely hope that this experience provides something fun for both new and veteran players.

In addition, we also have some more fun stuff cooking up behind the scenes which we hope the community will love, and we hope to continue growing as a community, and a gamemode.
*Thank you to Admin jiren74 for the amazing banner used on the site and post!*

Hello Survivors!

Since our last update, major changes have taken place across the entire network. The network has changed, with many new faces being welcomed onto the staff team and some other staff resigning. Lead developer, DocCodeSharp, was among those who have resigned.

DocCodeSharp began his time on Shotbow as a player and ended his time here as one of the most influential people on Shotbow. He came to MineZ and Shotbow at our lowest point and raised us back up. We owe him a lot. While we are sad to see him go, life must continue. The end of one’s journey, no matter how large and impactful, cannot be the end of our journey; so Shotbow goes on.

Now this change took some time for us to get used to, and any staff were called upon to serve new roles. Now that the dust has settled and our path is clear, our first steps can be taken.

So, what’s next for MineZ?

Future Development Plans

Previously, MineZ was fortunate to have one dedicated developer. Plans were drawn up for a large schedule of updates that were made public. Of course, only Hestril knew the progress on these, and many pitfalls befell us. Now that developer has parted ways.

Many devs will now be taking over for Doc, working together to bring a slew of updates. It is our hope that we will continue to churn out new features, bugfixes and quality of life Improvements at a good rate over the coming months.

This update is an amalgamation of many changes to the game – some big, some small. In time, we will lay out a new roadmap for you to follow along. For now, please trust that MineZ is in capable hands and we will do right by you.

New Challengers Appear

Previously released during the Origins update, Revenants and Husks were the first addition to the MineZ Zombie catalog.

Though, interaction with the entities of MineZ have always felt a bit bland. Having been influenced by other popular zombie survival games, we set out to add to this catalog.

Step 1

Teased some time ago, we needed to make good on the promise of Forsaken. They were teased some time ago and needed to be brought into the fold. These frightening, roaming zombies found in the north, are cursed. Forced to wander and given great, lingering power to enact a final judgement on any that cross its path, they should not be taken lightly.

Step 2

With the Forsaken complete, we needed to make this mob, and others like it worth finding and fighting. Unique mobs(those with nametags) now carry with them the ability to drop special loot. These mobs now carry with them a unique drop that should be pretty interesting and compel you to find them!

Step 3

With mobs and loot addressed, we needed to consider some mechanics that players utilize; zombie headshots, and bows to an extent. These are both valuable tools to quickly dispatch any pesky threats to your life, though you'll find not every zombie comes in the same shade of green now. You’ll have to get more creative, because these unique mobs can’t be killed with a headshot. They are protected…

So with that being said:

• Added Forsaken
• Added Scorcher
• Improved Revenant

Now you might be wondering….What is a Scorcher? Good Question!

These are only the beginning of the expansion of mobs we have planned to help expand the world of MineZ.

All Natural, Produced Fresh for your enjoyment

Now, I’ve wandered around the world for a good time, and I should note, this world is in the midst of a zombie apocalypse. What’s that you say, survivor? You don’t see any zombies out there in the wilderness? Well neither did I.

Fret no longer, coming to a wasteland near you, zombie spawning is back! Be careful where you hide, be careful where you look, natural zombie spawning has returned.

This has been tested, though it might prove to be too many zombies for a zombie apocalypse survival game. If that should happen, we can tone it down.

Prized Possessions deserve to be useful

We have taken our time to modify a couple of legendaries that have been highlighted as problems. These were either too strong or outdated. While we can not get to every legendary in one update, we hope that doing these legendary updates piecewise will slowly introduce more useful items to the game.

Truth Bow:

• Requires a full charge
• Consumes 3 durability per shot
• Shooting a player will mark and give them a glowing effect for 10 seconds
• While marked, the player takes 15% more damage from all melee attacks from players
• The shooter will be given Slowness 2 for 10 seconds

Binding Bow/Slow Bow:

• Consumes 3 durability per shot
• Slowness effect duration now depends on how far you shoot a player from, starting from 1 second (less than 5 blocks) up to 7 seconds (more than 40 blocks)
• Now does damage

Vampyr:

• Fixed an issue where the healing chance was incorrect
• Does 5% less damage than a normal iron sword
• Works against zombies. Healing chance is 15%.
• If Vampyr heals, it loses 2 durability.
• Vampyr now notifies the attacked player in chat so they can counter against it.

Hurter:

• Each use of the Hurter consumes 40 durability now.

Overkill:

• Overkill has a brand new effect.
• You can use it like a diamond sword, but also right click to charge Overkill's legendary effect.
• Charging takes 50 durability.
• The next attack after charging deals true damage based on the enemy's missing health. The more health they are missing, the more damage they take.
• The active effect lasts 5 seconds, or until you hit an enemy. Whichever comes first.
• After using a charged Overkill, all melee damage is reduced by half for 10 seconds.

Added one new legendary item.

Added two new Utility items.

A Hot Location

Remember that small little encampment north of Carmi, and south of Afaya? Did it start with an S...? Hmm. Nobody knows for sure. Either way, there is burning rubble there now, and nobody seems to remember what was there before.

In other news, a new location has been spotted in the now nicknamed Dungeon Desert. Check out one of the first photographs of this area below.



Indeed, Saens has been rebuilt and relocated to the desert just north of the Tristitia forest. Built by community member EnderMCx, this beautiful re-imagining of this old location was a community build we just couldn't pass up. Special thanks go out to EnderMCx for this build!

Now, if you're wondering why the image is so dark, well, we just want you to go explore it for yourself! Where's the fun if we show you all of the details before you can experience it yourself?

A Quality Experience for Quality Players

Now in addition to a wide variety of reworks in the code, we have been targeting to release more quality of life features and bugfixes for some time now. The team has been working very closely with players to note what features and changes would be appreciated.

• Axes now damage all non-player entities. We'll see how this plays, and adjust as needed, but this one made a lot of sense to us.
• We have modified Paluster Shrine again, this time more closely aligned with what works well for players. The changes are light, but should help!
• We have removed Abandoned Mineshaft, and promptly readded it as a Tier 1 dungeon, fully rebuilt. You may find that it has also been relocated to better fit the Tier 1 ranking.
• Arbreton and Geuten have been relooted. A handful of other locations have had some loot altered. More information in the changelog.
• Fixed a couple of duping bugs and other potentially dangerous exploits.
• Ghost Ship has moved again! Good luck finding it this time!
• Spawnkit Speed Potions and Grenades can now be used in the south again. We'll see how this feels!
• Horizontal Grappling has been made a bit more strict. Grappling at a height of 4 blocks will now count as a horizontal grapple.
• The potion chests in Al Hasa can no longer be "griefed" by standing near them to prevent them from spawning. The respawn timer has been increased to compensate.
• Bags can be opened while looking at the ground now.

In Closing

"In three words I can sum up everything I've learned about life: it goes on" - Robert Frost

While the past couple of weeks have been quite strange, we have emerged with a redefined team. Once more, we have committed ourselves to transparency, communication, and an overall increase in fun and exciting features to MineZ. We yearn to make the game interesting to explore, to fight in and to survive in

We remain committed to making this game for you, and will continue to do so as long as we can. Looking to the future, October is only a month away. What sort of surprises may we be planning for you? Well, we have a ton we are planning for this Spooktacular Halloween and we'd love to share some of this with you in our last thread of September(coming soon)!

But for now, as always,

Thanks for flying Shotbow!