Justin Glibert: Ember, and building onchain games.

Sina [00:00:19]: Hey everyone. Welcome to another episode of Into the Bytecode. So fun fact, the conversation I'm publishing today is actually the very first one I recorded for the podcast about six months ago.

It's a conversation with Justin Glibert about an early prototype of a game he was working on at the time and which he's close to publishing now. We got to talk about some of the patterns he uncovered while building Ember, which is the game, and Lattice, which is the engine behind the game. Patterns like inflation and zero-sum resources, spatial constraints, and user impersonation. Justin is a friend and a brilliant person. I hope you enjoy this conversation as much as I did.

Justin [00:01:03]: The interesting thing is that when you build something that is fully on-chain-- so when you only have a client and then choose and it is just on-chain, the interesting thing first is that-- you know when you build video games that are multiplayer there are two big paradigms. One is server authoritative. So the server is basically just ticking the game internally, and then clients just send commands and clients just try to optimistically predict what is going to happen and they roll back if the server just sends them something that conflicts with their internal state.

You need to build authoritative servers now because otherwise, people will cheat. There is no way to prevent cheating with authoritative servers. An example is with Minecraft. Minecraft is not authoritative. So Minecraft is authoritative for blocks that are broken or not, but as an example, the Minecraft server doesn't take the physics, which means that the client-side in Minecraft you can fly.

There is no way for the server to make sure that the jump you made is valid. So there is a very simple protection service side. It just checks how fast you're moving, and if you move too fast, it will ban you. But apart from that, if there's a jump you can barely make and somehow you'll slightly change the physics content such as you can, there's no way for the server to know because it's not authoritative. Whereas games like Overwatch and League of Legends are completely server authoritative because they're just way too competitive.

The annoying thing when you build server authoritative games is that even if you architect it really well, most of the time you have to implement everything twice because you have things that are client-side and server-side. But for origin stuff, it's pretty interesting because they're authoritative by default. So the way you do it is you write your contracts with the rules of the game, and then your client just updates the state through events, muted the state through transactions, then that's it. You have several authoritative games just from that.

The other interesting thing too is that if you build a game, it's really hard to show someone what is the critical part of the code that has the rules that needs to be audited or verified because it's all over the place. It's tens of thousands of lines of code because the rules are mixed with the business logic, with the networking layer, with the icon management. Whereas here, it's just the rules. You look at the smart contracts there is nothing but the rules. So this is pretty nice because at any point in time-- the rules of all game are already not complex, but have a bunch of edge cases that we had to implement. At any point in time, it can just open a Solidity file that is 400 lines long and I have the rules of the game. So this is already very interesting, I think.

Sina [00:03:35]: So is part of how you think about the game design to minimize the rule sets that are in the smart contracts and then just leave the client-side completely free? Like, leave degrees of freedom that people can play around with client-side? Do you proactively think about that?

Justin [00:03:51]: Yeah. Writing Solidity is hard, and the limits of the EVM are very restricted. So the way we think about it is anything that can be client-side should be client-side. As an example, the smart contracts will expect you to send things that basically does most of the work already and then it just does checks on top of this. But as an example, if you play the game, you'll see the bunch of tiny workers going around and stuff like that. Those are completely client-side. They do not even exist on-chain. They're just like rendered in new clients. Monsters are in regions which are like in areas of eight by eight squares or tiles. When you play the game you see them moving around in the region, but it's on entirely client-side.

So basically, one thing we discovered is you can make the game feel more alive and interactive by just adding-- basically sugar coating client side. Similarly, the client-side you can queue a lot of actions and create arbitrary complex. We are building this thing now which is called the meta action queue because right now there's an action queue. But what if, like you play an RTS, sometimes you want to drop rally points. So every time something pops out of your Barrack, it should move there. But right now we only have transactions to move from one region to another. So now that we have an action queue, we want to be able to have a Lua entrepreneur in the game so you can build Meta actions on top. I want my monsters to patrol around this area.

So that patrol would just be some Lua code that just add transactions to the queue based on a specific pattern. But all this is pushed client-side because ultimately the contract is just verified to the physics of the game and the inference are now broken and it does all the authoritative stuff. The client just syncs the state and prepares a bunch of transactions. I built a bunch of games before that are multiplayer and this one is the cleanest in terms of implementation because of the restrictions we have; by building on top of Ethereum.

Sina [00:05:45]: Obviously the gameplay experience matters. So if someone's actually going to enjoy the game, the sugar coating layers on top, and these monsters that don't actually matter at the roots level, play a key role in that person having a good experience. But on the other hand, if these games, especially being on chain, you can take the NFTs out and sell them. There will be people who will just be kind of focused on what actually matters here and they would probably create skins that just get rid of all the useless stuff. It maybe is just the terminal window that is solving math problems. Do you think these two groups-- and there are other types of players too-- do you think they'll coexist in the same games and how will they interact?

Justin [00:06:33]: Yes. This is a really good question. Like at Dark Forest, George Rose from Paradigm just implemented the Dark Forest internal part of the gaming in Rust. He said that next round his empire will be algorithmic. He's probably going to destroy everybody because people write plugins that are client-side, but they're slow because the game has a bunch of performance issues. He is just re-implementing the whole gaming rust. It is going to run on bare metal and it's going to be so fast. So we'll see.

It's going to be interesting. I think that the problem right now, as an example in Dark Forest, depending on your leaderboard positions you get them into the NFT. People started putting pretty high bids on those NFTs, which is kind of interesting if you think about it. It's kind of like buying the price. If you were to go to the Olympics and instead of competing you were just to buy the price from an athlete-- I don't know what I think about this, but people are trying to do that. So it is true that they are going to be very different stakeholders and people that are very different utility functions when they play the game. So one way I'm thinking about is those players did coexist in games like EVE Online and World of Warcraft.

Sina [00:07:44]: There were players who were just leveling up characters and then selling them when they get to like level seven.

Justin [00:07:49]: Yeah. So either we can figure out what their playbook was to make this enjoyable for everybody. Or maybe, for them, the problem would've been worse had they not gone through the effort of having all those empty chain solutions and banning the gold farm in China and stuff like that, because here, there's no way for us to do that. If you build a game that runs on smart contracts you will have bots. So I think that it's less about what can we do to prevent this and more about what kind of mechanisms design we can think about to just not incentivize those behaviors. This is really hard and I don't have an answer yet. Do you know Ultima Online?

So Ultima is one of the first RPG. Ultima Online was the first graphical MMORPG. Interestingly, any game that has been built after that, like World of Warcraft and stuff like that, it's all based on Ultima. They created the patterns and now everybody is just implementing them. They had very interesting problems because they really wanted to let players be player killers. So they didn't want to safe zone at the beginning because they thought it would prevent people from actually role-playing properly because who is safe anywhere in the real world. Or like, even in Fantasy, in Lord of the Ring, you can't just go into an end and then you're protected from the millions of sores. You can't do that. But they had a pretty serious problem with this because turns out killing all players is way more enjoyable than doing anything else in the game.

They never actually solve it. The game probably died because of this in inflation, which is why yesterday we gave a talk about ticket engine. I was talking about the fact that some resources are zero-sum because inflation is a big problem in virtual words. So right now I'm spending quite a lot of time just reading through postmortems of those games, trying to understand what went wrong, and how you can build to prevent this. But I don't have any satisfying answer yet.

Sina [00:09:44]: I feel like there's this interesting parallel almost like with the professional kind of amateur division with staking on Ethereum. Where you have these large validators and the protocol has been designed to incentivize small people to run validators. It's actually, if you violate the protocol and you're going to get slashed, the amounts that you get slashed is proportional to how many other people in the network also got slashed for the same reason. So if you're in the same pool; in a large pool, and that pool gets slashed, you get more damage than if you were uncorrelated in the way that you slashed. So the protocol has this decentralizing influence on how it evolves. I think this idea of inflation in the game and zero-sum resources is pretty interesting. How does that work?

Justin [00:10:43]: So the interesting thing is that there are sources and sinks in the game. The sources in the game are, you can mine gold and you can create it from time using gold generators. It's pretty dumb but it works for now. You sinks syncs because you can build buildings that cost gold and it get destroyed. So some value gets created and at some point, it gets destroyed. Basically, you want to balance the sources and the sink. But the problem is that there is no way for you to make sure that it checks out. It's really hard. So as an example, in MMOs the way they do it is, let's say you are a player who sells dragon skills. This is like you role-playing is a dragon killer. You have everything you need to kill dragons then sell those items. But if the price of dragon skill is dropped by so much that now people are actually buying them from NPCs, because it's so cheap, then your entire identity in the game is shattered.

Sina [00:11:43]: Because you had oriented around.

Justin [00:11:45]: Yeah. Now it's not profitable anymore. Then to give the regulatory good, "Just go fishing. Fishing is nice. Fishing makes money." But you don't want to go fishing. This is not what you want to do. So this is why interestingly in most games, there is basically NPCs buying items at a certain price and selling at a certain price. So those provides boundaries on how expensive and how cheap items can be in the player-generated economies.

Sina [00:12:10]: Like the assets are backed by some authority that will always purchase or sell them to you at that price.

Justin [00:12:19]: Exactly. But it's ridiculous if you think about it. There is no seller in the real world that will happily take your million dragon skills at a certain price without everyone changing that price if you have sold them like 500 K. It's just ridiculous if you think about it. But this is how those economies work. This is what they had to do to make it not completely free market and to generate completely.

Sina [00:12:41]: So there's no kind of notion of supply and demand once you get to that price.

Justin [00:12:45]: Between that, you can. But between those two boundaries, the market can be volatile. There are macro events that will drive the price. But if it saturates to the upper bound or on the lower bound, then yes, the market becomes boring because players can just buy from the house essentially or sell to the house.

Sina [00:13:05]: So this is like monetary policy?

Justin [00:13:08]: This is monetary policy in virtual words. There's a really interesting book about virtual economies. I'll teach you the title later. I'm reading that right now and it's fascinating. Did you know that Tencent-- it's like WeChat in China, but the whole payment system started because they had points in the app that you could use to buy cosmetics? Then people realized it was better money. So everybody was paying each other in Chinese using those points because at some point you could get it out of the system for real dollars. The Chinese government or Tencent you basically have an alternative currency, so they had to shut it down and back it with Yuan basically instead. It is crazy because if there is any store of value, people will use it.

So the problem now, because you have the whole interoperability with the rest is that you have to think about it. Is it possible? In Dark Forest, you cannot get your energy and your silver out of the game because otherwise, people will just start creating them. It will just happen. And the premise then, your game becomes speed to win, if this is important. So what you have to think about is, "Can you trade those resources? Can you pull them out of the game?" The way we're thinking about this right now is actually in Dark Forest, you can break causality. What this means is you spoon somewhere and you have to expand your universe progressively at a certain speed and you grow. But the problem of this is whether you can give to clients. So a really easy way to break causality is you spoon in one place in the universe, you spoon in another place, you gift the planet from the edge of the universe to the first account you have, and then you're in two different places and you are essentially teleported.

Sina [00:14:40]: Interesting. So this is why if you join the game with two whitelist keys-- that's how you teleport?

Justin [00:14:48]: That's how you teleport. The problem is if you could withdraw money from the game and then put it back, you could teleport too essentially because you withdraw it in a certain place in the universe. Then it's back into normal Ethereum land where the whole universe is compressing to one point, there's no notion of distance and then you drop it back. So one thing we want to do is we want our virtual goods to be interoperable with the rest; not like in Dark Forest right now. They might change it, but right now it is not. We want people to be able to trade our things. But we want to make it such that those assets will refuse to be traded if the trade happens between contracts and addresses that are not in the universe and not close enough in the universe. When you do that you still cannot break causality. Because there is no more notion of withdrawing from the game-- the things can always be moved away from the game, but it would only allow to be transferred with a contractor, with another person, if you're close enough. So here you couldn't do the teleportation trick.

Sina [00:15:43]: It's interesting. As you're talking, all these parallels come to mind for me with like the rest of crypto or the rest of the world where, for instance, there are certain security tokens. Tokens where, if a musician is selling rights to their future royalties and whoever holds this ERC 20 token is entitled to a piece of those, those are considered securities. So you can't just go on Uniswap and sell them to whoever is on the other side. There's this project Harbor that had built this R-Token model where, inside the ERC 20 transfer function, it would basically do this call out to a whitelist and see if the recipient address was also on the whitelist. But this is generalizing this idea to you call out into this distance function, and if the address is within some distance they can receive. That can be any function. There's a lot of expressiveness.

Justin [00:16:52]: It's really interesting. A bunch of distance functions are very abuse. Like if you're in a grid and how far are you from your turn in the grid. But it can be an embedding space or a reputation space. Are you friend with each other like in a social graph? You wouldn't be able to transfer things to people that are too far from you in some sort of social graph. Anytime you have a distance function, you can embed any sort of it. As you said, it is super-expressive. I think this idea is very interesting also because it greatly reduces the arbitrage problems. Let's say you just watch the Man Pool. There's an arbitrage opportunity in a cavern between a merchant and a player. Well, if you're not there, you can't do anything about it. Arbitrage bots would have to be at every single place in the universe to be able to do arbitrage basically.

Sina [00:17:47]: Well, again, this makes me think of the real world where there are these exchanges. There was the exchange in Korea or in Japan and the price of Eth was like 5% off from the exchanges in the US. You have to go set up legal outposts in both countries--

Justin [00:18:06]: Be able to exploit arbitrary.

Sina [00:18:08]: Yeah. It's interesting. On Ethereum, on open blockchains, by default, everything is available to everyone. It's one pool. Things tend in that direction of unification. The fact that you have one Uniswap pool for a particular pair means that the liquidity builds up. But here we're talking about artificially creating these distances and separations and seeing what sorts of things emerge.

Justin [00:18:43]: I think it would make it more fun to be fair. EVE Online is mostly economics. People play to basically learn real life and business. Players like it because it is inefficient. Did you know that people trade derivative in EVE Online? They wake up in the morning at 8:00 AM and they open their leverage dashboard and they trade derivatives in titanium and on whatever spaceship parts. It's crazy. But people like it because the market is sophisticated, financially speaking. But at the same time, it is still possible for parrots to crush the prize by destroying a big cargo. So it basically mixes seriousness and playfulness together; if you consider finance to be serious. So I find that's fascinating and I think we could just literally do the same thing. But with the whole added benefits of internal probability where it's not even possible for us to predict the second-order consequences of things. Whereas in EVE Online, everything had to be designed by that studio in Finland.

Sina [00:19:48]: Yeah. If you change what you're optimizing for from just purely being what's most useful, and change what you're optimizing for to what's most fun or playful, the space to explore there is huge.

Justin [00:20:03]: Yeah. You talked about that at Eth CC, right? The first production-ready thing that happened is DeFi because given gas costs and fees, the only thing that makes sense is making more money than the fees. DeFi is objectively easy because when you build any financial application, the difficulties like maintaining a proper ledger, double-entry accounting, security, and stuff; but for Eth, you get all of this out of the box. Afterwards, it's just like, "Can I implement math to make those financial protocol work?" And they're easy to implement. I think that games are the second step after that because they're harder to implement, but they're still fully on-chain. So you don't have all those nasty problem solving to deal with off-chain stuff.

With DeFi, people could not have predicted what happened if they just tried to look at the traditional financial systems and then adding decentralization. It was not possible to predict what would happen. DeFi Summer and Yield Farming is just completely emergent. It makes no sense. I think similarly with games right now, we just think it's going to be like a simple reward or Roblox, but decentralizing people who write scripts on top. I think that this is the case right now, but in the future-- like maybe a year from now, it's actually impossible to predict what is going to happen. Bunker stuff could happen.

Sina [00:21:23]: Yeah. We're constantly surprised by what people do. So one of these ideas that you were talking about yesterday that might be the seed of something emergent was this idea of impersonation. What do you mean by that? What does that look like?

Justin [00:21:39]: The way we reached that concept of impersonation was mostly for security/ engineering reasons, because the way you play Dark Forest and you play the way you play ZK Dungeon is you don't connect through WalletConnect or MetaMask. You have a burner wallet that just approves signed transactions and send it on-chain because we want to be able to send a lot of those transactions and having a popup every time would be annoying from the user. But we also want people to be able to connect to ZK Dungeon and Dark Forest with the same identity in other games too. So right now if you wanted to do that you would have to copy and paste your private key on from your burner wallet.

Sina [00:22:15]: The burner wallets in your local storage are specific to the particular--

Justin [00:22:20]: Exactly. Ultimately, you don't want apps to do key management for you. You want to be sovereign of those keys. So what we thought about is, your identity is the address derived from the private key that is inside MetaMask or your ledger. But then from this account, you basically authorize a burner wallet to impersonate you in the contract for low stake actions. So for everything but transferring NFTs, that burner wallet can do actions on your behalf. But the interesting thing is, as soon as you have an impersonation mechanism, you get a lot of features for free out of the box. First, you get GUILD. What is a GUILD? A GUILD is a smart contract that joined the game as a player and then approves a bunch of players to play as the GUILD. Then those players can also approve burner wallets. So basically from that simple pattern, you get GUILDs. At least GUILDs in the simple form where they share their treasury, their units, and everything.

Second thing you get is if you can set a scope-- Do you know 0Auth in Web2 has scopes for what apps can access? If you can set scope two with that impersonation pattern and because it is interoperable, you could actually rent access to some part of your dungeon to people. You could be like, "You pay me that much. Or you do this." And for the next 12 hours, you can control those monsters. Then they automatically go back to me. They never left my inventory. It's just that you can do stuff with them on my behalf. So just from that simple impersonation pattern, you can already get a lot of mileage through it.

Sina [00:23:52]: This last thing makes me think about how certain YouTube channels-- I was just watching this thing. I don't know if you know this channel, "Yes Theory." It's pretty fun. I think their videos are cool. But they have done this thing a couple of times where they ask their audience to choose what they do. It's like a ‘choose your own adventure.’ You could codify that into the rules of the game where you literally hand over the reins, the permissioning to some user--

Justin [00:24:25]: To watchers on Twitch

Sina [00:24:26]: Right. And they could go get you involved in some crazy battle and then you are this world-class player who has to come back and just dig yourself out of that. How much more engaging this would make the game for other people who are watching it.

Justin [00:24:45]: Yeah. The craziest thing is we would get that out of one morning of interface design and solidity. It's literally five lines of code and some UI for this. Whereas in another game, there are companies that are entirely built to create products that allow traditional games to have those sort of things on Twitch, as an example. Like, "We have an SDK so that streamers can let their users control their character." We get it for free because of interoperability. I think we're going to see this happening over and over again, where we fought the hard battles at the beginning and with all those constraints, but now we can move much faster than anybody else. We already have a scripting language; it's solidity. We have anti-sheet authoritative servers, account management. All this is just there for us. We didn't even have to build it. So I think it's pretty insane.

Sina [00:25:31]: You have a very small solidity contract that enforces the core ruleset of the game. Anyone can write other things on top of it; whether there are other smart contracts or web applications, and then you have this whole ecosystem potentially building out this game. What do you think the evolution of these games will look like and when they'll become compelling to the general game-playing public? It seems like they're going to go up this curve at some point. These improvements and the ecosystem is going to compound on itself in ways that a closed game with even an API won't be able to match. How are you thinking about this in your own plans?

Justin [00:26:18]: I agree with the last one. I think that the second or the third derivative here is extremely high, but we don't see it. So it's going to look like nothing and a toy for a long time and it will start compounding so fast that we'll barely keep up. But it's fine. I think one thing that is interesting when you build crypto app is you just give it away to people and then they do things with it and you finally can could sleep at night, because if things have been well designed, then they should not be on fire the day after. So to go back to the question, I think that the way we're thinking about it is right now, given scaling issues and just the general immaturity of a lot of tools, I think we will probably have to stay around enthusiasts for at least a year or two.

We're seeing people that know nothing about Ethereum and even blockchain play the games because they find it interesting. One thought we had from players today at the land party is that they thought it was amazing that they were playing a game that nobody had figured out yet, because a lot of traditional games have been min-maxed and completely analyzed and have been figured out, whereas here. This is not a property of what we're doing. It's a property of any new game. But there is something for players that are traditional gamers.’ It is a new game that people are playing.

I think that another thing I don't think I found really interesting is that people said that they have way more "of the game experience" in those games than in traditional games. What I mean by this is in traditional games you play a world garden that has been completely designed by the game designers. Then the only out-of-the-game experience you have is talking about it on Discord. There is no way to do stuff that involve the game apart from communication. Whereas people who played the Dark Forest round said, "Wow, I bought hash from a shitty guy on Discord. I sold my maps and I had strategies." If you think about it, this is more the metaverse than the game itself. The metaverse is you LARPing on Twitter and on Discord and moving things around. This is the experience that matches what has been described like Snow Crash and Player one. It's not the game itself.

People are too focused on the games themselves. The game themselves are just shelling points to bring people together. But the actual metaverse is all the interactions you have around and all the stuff you do. But for traditional games, the maximum thing you can do is just post things about it, read it, and talk with people on Discord. But now you can build on top, you can share things. You can do that to an extent with Roblox, but it's pretty limited in want. I think the fact that they have all those out-of-the-game experiences is also very interesting.

Sina [00:28:54]: Another idea in the vein of why people find these games fun to play is that it's at the forefront of using this new technology. You could look at ZK Dungeon or Dark Forest as just the game itself and you could build the whole thing. The back end of it could have been just straight-up servers. The gameplay might have been exactly the same for all intents and purposes, but there's a design decision that you've both made to not hide that away. Literally, put the fact that these ZK proofs are being generated or verified and there is a freaking terminal on the screen when they don't need to be. But that's part of why people find this exciting.

Justin [00:29:46]: It’s just that at the beginning, projects are always default death. They're not default alive. To turn projects from default death to default alive is you want things that are extremely polarizing at the beginning. Most people say, "Nah." But the remaining people are just, "Holy fucking shit! I just found my tribe." I'm just so happy to see those proves being proved. And the fact that when I'm playing, it's basically state-of-the-art math that is used. Another thought I had when I was playing Dark Forest and ZK Dungeon was like, Wow. Blockchains are probably the most resilient data layers ever. And my dungeon would probably be around 30 years.

Sina [00:30:27]: Like this game, I'm playing this evening. The artifacts from it will literally be around after I'm dead.

Justin [00:30:34]: Yes. Most probably. It's insane if you think about it. Just because it's probably the most durable data storage in existence. Same thing with Bitcoin. Bitcoin would be run until the heat death of the universe, most probably. It's just as you said because you embraced the fact that it's technologically at the forefront-- it's just bleeding edge. People played for that. But I also think that people would play it even if it was not just for the fact that it was bleeding edge. It is kind of contrary to this.

But I think that the interesting thing here is people play for the community because the game attracts people that are interesting. So let's say you have profile A that wants to play things that are at the bleeding edge; who plays the game. Then you have profile B who doesn't care about the bleeding edge but cares about being around people that are interesting, will play because of this. So it's kind of like in nightclubs where at the beginning you start to get really hot people in the nightclub so that afterwards everybody comes. It's the same thing here and it works.

Sina [00:31:40]: Yeah, totally. That is a really good point. It's like the same metaverse layer two of the game-type idea. It's why it's just fun to be an early adopter of things, even though they probably suck to use at the time. But the community that's forming around them is what's special.

Justin [00:32:00]: Right now, if you were to make a social product or a social app that purposefully sucked, nobody would use it. Even the ones that don't suck people don't use it most of the time. But here it's different because there's this sense of belonging that people can have. Every time I think about how bad the game look, I think about Dwarf Fortress which is literally an ASCII game that has just a very fanatical following just because it is interesting and hard to play. They identify with the fact that they can play a game this hard to play and the fact that they likely belong to this community. So I think this is how you bootstrap. You start from those very polarizing features, and then progressively, you try to retain this group of evangelical users that are being polarized at the beginning while progressively appealing to a bigger Venn diagram; more and more mainstream.

I think the difficult thing here is at some point you start losing the evangelical users who would then just ford the game and make it hard again, most probably. So this is the difficult part I would say. We've already saw it happening with Dark Forest, actually. Some of the players that were playing in the first round just gave up and are not really playing anymore because now it's like they were playing and it got too serious and people are playing for the prizes and stuff. So at some point, as you will, as your game will evolve and your community will evolve, you will drop some of the demographic automatically.

Sina [00:33:17]: It's part of the evolution. It makes me think of how people are at Burning Man. Burning Man stopped being cool 10 years ago. People have been saying that since it began.

Justin [00:33:30]: Well, because you start with actually really, really weird people. It goes less weird over time and then--

Sina [00:33:37]: Totally.

Justin [00:33:37]: It's probably the same thing.