"You're telling me there are game development studios in Egypt?!" - Some guy To the beautiful melodious sounds of a Madisen Ward & The Mama Bear song, on an expectedly very hot Saturday afternoon, the fan trying its best to shake the stale air, I sit on my bed looking back on some nine months I spent with Instinct Games. Fighting the urge to just write "Instinct yohoo!! Ark yeaaaah!! GET HYPE" along with a dank meme or two and be content with myself, and risking one or two looks of derision from some readers sceptic of my objective look at the company I would be wise to subtly compliment, I will say that Instinct Games is one of the biggest game development studios in Egypt. It has some of the most highly accomplished game developers in Egypt, who worked on titles like the Reality Engine(purchased by Epic), Cell Factor: Psychokinetic Wars, Dungeon Defenders, Playverse, and of course Ark: Survival Evolved. Fresh out of computer engineering college, a young buck from an inconsequential city, I had almost no real game development experience under my belt. Perhaps with the exception of a Real-Time Strategy game as a graduation project, in which I worked on AI, and a small contribution to the open source kart racing game SuperTuxKart that netted me a mention in its credits, which I hold dear as my first ever. I was aware of my inexperience, and had applied to only a few game studios. I got the call from IG while I was getting a ticket for forgetting my driver's license, which has now become a very memorable and happy moment. I've been on the Ark team for a total of 5 months and in the professional game development sphere a total of 9. A very short period to have anything worthwhile to say, unless this is a very calculated and far-thinking move to prepare for the performance review coming in just three months. Joking aside, the quick transition from thinking what game should I make to improve my skills and portfolio as an unemployed fresh graduate to the fast paced life at IG, the allure of working on a hit game like Ark, and the extremely valuable knowledge I am constantly learning at Instinct, are the driving points for this article. My love for writing and documentation also play a role. There are four main titles in this post: Unreal Engine 4, Long Hours, Yak Shaving, and some notes on being part of Ark's community. Categorically, we can say that UE4 and Yak Shaving are more geared towards programmers while the Long Hours and the community notes ones are for anyone in the game development industry. Feel free to jump to the parts that interest you. Unreal Engine 4 Mummford & Sons's Dust Bowl Dance now. Describing my time as an Ark developer(an Arkeloper? No? I'll show myself out) as tumultuous would be most fitting. I had no experience with Unreal Engine 4 prior to starting and being fresh out of another episode where I had to quickly learn Unreal Engine 3 for Qube:Director's Cut, I approached UE4 a little differently than I had done with UE3, which I had watched many tutorials for, made some prototypes of my own, and followed an fps tutorial as well. This led to me cramming a lot of info about UE3 that I never needed and never got to use, and ended up forgetting eventually. Studying up for UE4, I watched only one video, preferring to read some of the introductory docs and fiddling with the editor myself, learning as I go about my tasks. My first task was to modify the behaviour of flying dinos, e.g. the Pteranodon. It was the first time I heard the term Behaviour Tree, so I decided to read up on it independently of UE4, which implements its own event-driven behaviour trees. This was quickly decided after reading this statement at the beginning of the UE4 BT docs: "There are three critical ways in which the UE4 implementation of Behavior Trees differs from "standard" behavior trees:" I figured it's only logical to know what the standard ones were before learning the event-driven ones. I found many useful articles, but this one on Gamasutra was the one that helped me wrap my head around the concept regardless of implementation. I went back to UE4's docs afterwards, which are quite well documented in BT's case (I'm of the opinion they are disappointing/insufficient in other areas, but I don't hold much significance to it because it's often much more beneficial to understand the logic through code, rather than searching the docs). You may already know of UE4's Blueprints, which are a relatively easy way to develop functionality in your game without any coding skills. Many developers dislike BPs. When they get big enough, they are much harder, and more time consuming, to debug. Seeing as my task was to debug someone else's BTs and BPs, I experienced this first hand. There was a problem with the follow behaviour of flying dinos, and I was supposed to find it and fix it. Relatively easy, in retrospect, and very wise of the development lead to assign me. Some Ark players reading this may be jumping on their chairs yelling frantically "There is STILL a problem with the follow behaviour of flyers you pixelated thing!!", and that is true, I am pixelated, but GameDev insists on a 50Kb pic, so what's a man supposed to do, take a new picture? (Are you reading this article on GameDev.net by the way, or have I become impatient and published it somewhere else and forgot to remove this reference?) Starting my relationship with UE4 and Ark with behaviour trees had opened my appetite for more. I sought to have a diverse array of tasks assigned to me, always trying to dabble with something new. My seniors knew this, and encouraged me greatly. In one milestone, I think I had only done one task by that time, there was a feature I really wanted but didn't get to do. The Brood Mother attack AI. The Brood Mother is a gigantic boss spider in the game. I'm terrified of spiders but I'm fascinated with AI, so I was drawn to it so that I would get to explore BTs even more. (Yeah, they did the let's-put-a-bunch-of-spiders-in-Yas's-TestMap-and-bunch-up-the-terrifying-sound pranks on me, it was hilarious guys, ha ha ha). I was young and hungry for experience. Work was and still is not work for me, it's the place I do immensely fun and cool things at. The task went to someone else though, and fortunately as well, for not only did Brood Mother turn out to be more terrifying than I could have handled, I still avoid the caves in the game as well(Sorry A.Q.Fatcamp, I let you go in there to die alone), but the feature developed into a much bigger one and the attack system ended up getting revamped and redone in a more scalable way. Long Hours It's not uncommon at all to stay in the office until well into the morning. In the Yak Shaving period I will get to shortly, we would stay until seven in the morning many days. In many days when there wouldn't even be a time issue, you'd find yourself staying until 2 or 3 a.m to 'just do this little thing'. Staying for two or more days at the office isn't very rare either. I don't necessarily view all of this as a negative aspect or a downside of game development. I am young enough, and this is new enough for me, to childishly enjoy the adventure so long as I'm doing something I love. It's an expected part of the experience, and I'm well compensated for it (But feel free to make me exceptionally compensated though, wink wink). We have to realize though that I certainly view it a whole lot differently than, say, someone with a family, children and a wife that they have both material and emotional responsibilities towards. I would also imagine it would get much more tiresome and less of an adventure as you get older, or if we had to work on a less exciting game. So far, I mind not being able to solve something far more than staying up late. Yak Shaving To the tones of Bad As Me, by Tom Waits. Developers have an unwavering trust in the absolute logicality of the machine. Because we know that, we know that everything is simple logic in the end no matter how convoluted it seemed on the surface. Faced with an illogicality, we know that either our own logic is wrong, or one that we are using either has a problem or is functioning in a different way than we had assumed. More often than not, it's our own logic that is faulty, so we are always advised to revise it again and again. When there's a problem and you cannot find anything wrong in your own code (you brought in your seniors to check your logic and they too cannot find anything and all), that's when you hear the phrase: "Well, you'll have to get to the bottom of it". It may sound like a no-brainer. Someone might say you don't need to be told that in the first place; you always get to the bottom of it, right? I think that's wrong. You only get to the 'bottomest' you can possibly get within the confines of your own assessment of where to stop looking. That assessment you estimate from experience, knowledge about the tools you use(i.e. your engine) and your own debugging prowess. Because you are not free, your time isn't your own and you have timed requirements, you cannot risk Deep-First-Searching every possible path the bug could have occurred from. You are, or should be, wary of delving into engine code too much, and you need something more than a hunch to start looking into what could cause the problem you are having in your game, from the engine code. When told the magic phrase though, you are relieved, you can debug to your heart's content because it seems the problem isn't just a simple mistake in your code. You can finally shave that yak. You're Loki, finally unshackled and ready for Ragnarok-level epicness (Get it? Epic makes ue4, and this will be epic. Such joke). This happened in my last task, which is my most finger-wigglingly entertaining and redbull-abusingly frustrating feature to date. It is the long awaited Collidable Saddles feature. In other words, build castles on Prontos: It is called Collidable Saddles because saddles weren't originally collidable. Quickly, with help from a senior dev, I hacked the saddle to be collidable through the editor. I proceeded to tackle issue after issue of building onto the saddle: Attaching to the mesh so it moves; attaching on server as well; figuring out structures alignment and linking; making the saddle collidable through code, like a proper gentleman would; saving the necessary data for loading from a save game; checking both server/client, and standalone; Fix some pesky displacement problems. Those were all normal issues solved by looking at game code and adding my own when necessary. It was smooth for about two weeks; I thought I was done. Then came the bug. When standing on the saddle, the player's character was jittering like a kid on a trampoline who just ate three spoonfuls of raw coffee. It was discovered it happened only after moving a bit with the dino, so the saddle is no longer occupying the same space it did when I first attached the saddle. Turns out the physics bodies of the saddle weren't moving with it at all. They get placed correctly on the client once you equip the dino with your saddle, but that's it, no more updating physics. This took time to figure out. The mesh itself moved with the dino just fine. I was frustrated, I knew there was something wrong with how I made the saddle collidable. There must be, I was so sure of everything else because every thing else had been straight forward enough. I decided to seek help. After reviewing the code, and understanding what I did, he(a senior) decided that we must get to the bottom of it. With the help of the Physx Visual Debugger, a tool another senior suggested we use to visualize the physics bodies, we discovered that the problem was that the physics bodies were not updating after being created. Further debugging revealed several conditions that were preventing the physics of the Slave Bones from updating specifically because they had a Master mesh. I run through this in a few sentences, but in reality it took days and countless unpronounced bondage related jokes to discover. This was pure debugging now, understanding the ins and outs of this part of the engine through the behaviour of its code. There were no docs to speak of now. The solution in the end? Force update the physics of the slave on a condition of our own making, so that we can guarantee the bodies are moving and animating correctly with the dino. Now, this is not the final implemented solution. A cleaner solution was introduced with the new building platform made for the pronto, which eliminated the problem partially, and allowed us to fix the rest using existing engine functionality. We shaved the yak, but decided not to use the hair in the end. There were many other problems and optimizations still needed that required some more midnight oil to be burned, but this particular one will register with me as a good learning experience. Community Notes: This is the final part and the prime reason for the title of this article. I love developing games. Games are, I believe, a revolutionary medium of expression and a very accessible source of therapeutic fun. We also have an amazing community of players that is super fun to interact with. They give me a whole lot of satisfaction and provide us with some really cool, and sometimes bat shit insane, things they made (Love the nude Mod, mismagoonz!). They have allowed me access to pools of experience I wouldn't have had otherwise either. The hectic early access launch day(s), the awesome tools some of the community has made for itself, and just watching the forums and reddit to get a feel for what people are concerned about or anticipating the release of. In this game I'm a small cog in a well oiled machine (admittedly some of that oil is midnight one, but hey, I ain't about to complain, I really love it when I can't find taxies home, it's hilarious. Walking is good for the health thing). Being the youngest in the development team, and least experienced, I get a good high from seeing the game grow so much, seeing the players anticipating an upcoming feature and seeing myself getting more and more cool things to make. It's like I, err, kinda, uh, grow with the game. Oh god, I'm so sorry, that was so RPG. I meant survive alongside the game of course! I'm tough, I tell ya. Yas.
The Incredible Lightness of Being An Ark Developer
Comments
More of an artist myself, but I always love reading about how you coders take on these challenges. Keep rockin it!
I remember waiting for weeks for buildable saddle platforms. It was worth the wait, if only foundations were not required. Foundations on platform saddles ruin the aesthetics and look really fugly.
P.S. Hope you guys are planning to resolve pillar onto pillar clipping before full release.
So the real question with regards to the long hours, is did you feel they were required to keep your job? Would you have been fired if you didn't work them? Would you have been unable to complete your work?Did you feel more effective working those long hours?
So the real question with regards to the long hours, is did you feel they were required to keep your job? Would you have been fired if you didn't work them? Would you have been unable to complete your work?Did you feel more effective working those long hours?
Let me tack on one more question to yours: Were the long hours necessary to get the work done?
I'll try to answer with not a lot of diplomacy. Let me first explain something subtle about the long hours. You stay late for one of two reasons: You have something that needs to be done on a specific date and you think normal working hours aren't enough; or you are plugged into something and you'd rather stay on it those extra hours so as not to lose your train of thought, which often has an undocumented list of things that you'll do. A quick example of the latter reason is what's on my mind right now as I type this reply. I have a few points I'm planning to address; I haven't written those points yet, they're just in my head, and I've not written a list of the points in some to-do list either. If I leave this reply unfinished now, when I come back to it later, those points won't be as clear then, and I may forget half of them. At work, it's a lot like that, to the degree that you postpone going to the bathroom so as not to lose that plugged in state.
We call the other reason crunch time. It happens when you're behind on your estimated time of completion, or about to be. In our company, crunch was never explicitly declared. No one comes to you and says "We gotta stay late until this is done" or "Don't leave before this is finished". At least from my personal experience and observation, this isn't how it works at IG. It's rather left to each developer's discretion and estimation of when the job would be done.
There's an oversimplification here though. Delays happen because of a wide array of reasons, sometimes outside the developer's control. For example, if you miss-estimate the time needed for something new that you haven't done before, and those are plenty, and it turned out to be more complex than perceived, thus requiring more time, that's not a fault of your own, is it? Why do you have to increase your usual working hours then? It's not your fault, the task just needs more time. More days. Regular, 8 hour, days. Right?
The answer isn't always yes, and, in my opinion, mostly even. There's a very important factor to take into account when trying to understand why some/most devs would feel obligated to stay late; Own-age (That is a word I invented. It is the noun of the verb own. I get to do that because I'm pretty cool). Own-age for the task, its completion, and of understanding how the system you built for it works and interacts with other systems, and own-age for the delay, even if you, as a mind, was blameless. It immensely helps, I believe, that there's no culture of blame at all here, and that, I have come to believe, is because your managers, seniors, and supervisors are also good developers who understand all of that above.
I think this answers all of the questions in some, perhaps needless, detail. For the last question about the work effectiveness, I'll assume you mean since it's more hours of work per day, your effectiveness would decrease during the last ones. That's true, definitely. The decrease propagates to the next day as well. If you work till 3 a.m. then go to work at 12 or 1 p.m the next day, assuming seven hours of sleep, and repeat, you'll be increasingly exhausted and drained. There are no general rules though, and you try to strike the balance during those, hopefully not long, stretches.
And, you know, redbulls.
Even with the owning of a problem you should never really stay in the office more than 10 hours, if you do this often enough it will dramatically decrease your performance. Also it kind of indicates that you can't estimate tasks or are trying to do too much.
I know the feeling of being in the zone but even if that happens at 18:00 when I should leave for home I hardly ever stay longer than 20:00 (I have to start at 9:00). And if I do this I will actually offset these additional hours within the same week most of the time.
Its really easy to pick up were you left off the previous day btw if you at the end of your day just write down a todo list of whats in your head at that point. The additional benefit of letting stuff simmer in the back of your mind is that you will come to insights you otherwise wouldn't have in most cases.
The problem with the youth full exuberance of doing this often is what fosters that bad name of the games industry for overtime problems. I have both done non mandatory overtime and mandatory overtime, both paid an unpaid, after three years of this I said enough and just outright started refusing even the mandatory overtime. The only way to fix our industries overtime stigma is to start refusing this stuff, so please don't glori0fy this.
Thank you for sharing your experiences so far! I certainly look forward to trying out ARK as well as anything else your company puts out. I'm just starting out myself and I think I may try what you did with UE4 and to learn Unity and go from there and it is nice to know that it is possible to get into this field with relatively little experience! And it is certainly wonderful that your team mates and senior devs were willing to work with you so much!
"In this game I'm a small cog in a well oiled machine (admittedly some of that oil is midnight one, but hey, I ain't about to complain, I really love it when I can't find taxies home, it's hilarious. Walking is good for the health thing)"
--A retrospective look at beginning a career in the Egyptian game development sphere.
I found this article awesome. I really liked that I could see how it feels to develop an AAA game as a junior programmer that just started. Thanks for sharing your experience.