Hello everyone! I’m Daft-Cube, and welcome to the first edition of Daftcube’s Dividends! This is a special devblog series that aims to discuss the development progress of the not-directly-spaceship-related-butstill-very-important gameplay and supporting systems of RoVerse! Daftcube’s Dividends are not like the current weekly devblogs because Daftcube’s Dividends will only be posted when new content is ready to be discussed. On the flipside, these devblogs will be on the longer side and will discuss systems with much more technical detail. If you enjoy boatloads of technical design commentary or simply want to understand the infrastructure that powers modern MMOGs, this devblog series is for you! Without further ado, let’s get started! Design Goals One of the defining features of an open-world space sim is trade. People around the galaxy are always in need of materials to build stuff, rare artifacts to make even more rad stuff, components for space stations, ammunition to throw at their “friends,” and more. Within RoVerse, we want to ensure that our markets live up to – or maybe even exceed – the expectations of the space sim genre. Our main design goals surrounding markets were…
After analyzing our design goals, we decided that a buy/sell order market model would be best for RoVerse. For each sector, players can post buy and sell orders for a good. A ‘buy’ order is an offer for the item in exchange of currency, while a ‘sell’ order is an offer for currency in exchange of the item. This system results in a free market where the interaction and competition between players determines the price of goods. It’s real economic theory at work, and it’s super cool! Our vision presented some major overarching technical challenges…
To see if these technical challenges could be overcome, we started work on several prototypes. Prototyping (Or, why failing faster is actually epic) RoVerse has a pretty large scope. In order to build features that seamlessly integrate and complement with the existing systems in the game, prototyping becomes critically important. Prototyping is the act of creating a small-scale model of a larger system to test if the system could work. Prototyping also allows us to discover potential flaws in our systems before they are built, saving us a lot of time in the long-run. During the prototyping stage of RoVerse’s MarketService, we first naively tried to create a system that used Roblox’s DataStoreService to post orders and propagate changes across servers. Roblox’s DataStoreService is what allows your favorite Roblox games to persist data across sessions. We thought that we could save market data like any other game data, and just have a server script take care of the transactions. The prototype went badly. Like, really, really badly. Implementing Realtime Markets, Take 1 (Or, why datastores should only save data.) We quickly ran into two major issues with DataStoreService. First, DataStoreService does not support transactional operations. Basically, when a player purchases something from the market, our code requires several different values to be saved in different DataStores simultaneously. We need to first remove the cost from the buyer’s ‘wallet’ datastore, then we need to add that same amount to the seller’s ‘wallet’ datastore. Then, we need to remove the order from the market’s datastore and add a record of the transaction to a ‘past transactions’ datastore. If any one of these operations fail, we cannot safely roll back the state of all the other datastores. The following chart describes what issues would occur if a market operation could not be rolled back after each datastore change. Second, DataStoreService is not built with high-frequency concurrency in mind. In simpler terms, DataStoreService has a lot of trouble when several different servers are reading to and writing from the same datastore very frequently. Specifically, DataStoreService’s :GetAsync() function ‘caches’ for four seconds between calls. This caused an interesting situation where if two players purchased from the same market order selling only one unit of a good within four seconds of each other, each player would get that item. Two items were sold, when only one was posted. Sounds like a fun replication bug that can be exploited! Some people might point out that we could have had more success using Roblox’s MessageService. This service allows servers within the same game to communicate with each other without creating a clever hack that used Datastore’s somewhat unreliable UpdateAsync() function. While we were prototyping, MessageService did not exist. Even now, it’s not a good option because it is still non-transactional and messages sent over it are not guaranteed to actually be received across all servers. Overall, we quickly discovered that the concurrency issues and the non-transactional nature of datastores was not going to work. We needed something better. And, we had the technology. Implementing Realtime Markets, Take 2 (Or, why a separate Javascript environment is more flexible than anything Roblox could ever give us.) How do we fix this problem? The resources that Roblox provides us within its engine are obviously inadequate for our vision. It looks like we need to reduce our vision and- You thought I was seriously going to say that? Nah dude, I don’t go down that easy! Roblox has a service called HttpService. This service does exactly what you might think it would do: it allows Roblox to send requests to websites outside of Roblox. A new design formed… We elected to build our market service on our own web server, and then communicate with it from the Roblox servers using HttpService. This method has many advantages…
Node.js is an open-source serverside runtime for Javascript. As I had just finished an internship that revolved around using Node.js as an intermediate between a database and an external service, I thought it was good enough for the job. We decided to use relational databases as opposed to non-relational databases because our team had more experience with the former. Now, our current tech stack looked (and still looks!) something like this: Our prototype was a whopping success, demonstrating that our concepts and tech stack were able to run a marketservice that could allow users to trade across-server while seamlessly rolling back all operations if an error occurred.
The Present and The Future For now, we remain busy at work building the market system. There are still many bugs and challenges to overcome. Some people may ask why I did not speak about buy/sell orders or wallets for groups. This is because that is still being figured out; we want to have organization-level market interactions, but we’re not sure if Roblox can handle it. We can only really prototype that once we have our existing market infrastructure up, too. We’re not sure what the future looks like quite yet, but we do know a few things. First, as the Roblox client/server development has been falling behind due to our Roblox developers’ real life obligations, it is likely the MarketService will not be able to be tested in the persistent universe for some time. However, we do want to test all of our web-based services way before then. We do not know how we want to do that yet, but rest assured you will get to help us test this crazy infrastructure in the somewhat near future. Questions and Answers! Queen Bishop Airyona of Luparis asks, “I take it a stock market is not possible?” A stock market is possible and could be implemented by creating another service like the MarketService that has an additional layer for dealing with securities management and all the math that entails. Realistically though, I don’t think we will get to stock markets. The markets themselves are a lot of work, and I think it would be best to focus our efforts on our current scope. Idyllic Explorer asks, “Taxes and Stuff?” Ah yes, death and taxes. We have solid plans on implementing market fees and taxes, however, we’re currently focusing on just getting the markets themselves functional in a production environment. After that, we’ll add taxes and its accompanying misery. In other terms, yes to taxes, just later. FangABXY puppy defender asks, “How does industry and crafting work again?” We’ll cover that in a later Daftcube’s Dividends installment! Idyllic Explorer also asks, “Will there be more graphs for the market?” I hope the diagrams in this devblog satiated your current thirst for graphs for the time being. In the final game, we hope to have a price history graph for each item. However, HttpService limitations may get in the way. It’s hard to tell before we have our first market mass test. Wait, did I say something? ;) [Oh god im pressed]Kronos asks, “In terms of the cloud infrastructure, how do we know that it won’t go out, and, if it does, how are you gonna remedy that while its down or if it stays down for an extended period of time?” Ah yes, downtime… Nightmare fuel… Part of development of any software is creating procedures for proper and improper operation. If the market server goes down, only market trade will be inhibited. Players will still be able to play in the RoVerse universe and directly trade items to each other. One of our design goals is longevity, so we’re hoping a longer-term outage doesn’t occur. If it does, us developers will probably have a horrible few days, but we should be able to get it back up. Thanks for Reading! Wow, we made it! That was a long devblog! Thanks for tuning in to Daftcube’s Dividends. If you liked this devblog, be sure to let us know! They take time to write, so we would love to hear what you want us to focus on. We’re not sure when the next installment will be, but we do know it will be waaaaaaay more technical than this one. Once again, thanks for reading, and here’s a teaser for making it this far! Comments are closed.
|
Archives
January 2023
Categories
All
AuthorsVaktus: Leader of the Vaktovian Empire and head of the RoVerse Project |