Beneficial Distractions

OK, so I have a legit excuse for my posting absence this time. It turns out the force directed graph (FDG) engine I have been developing to power the graphics of HTP has drawn some interest from the data visualization market. So myself and couple friends are spinning up a start-up company around some of this tech. It's exciting stuff, but has also torn my away from working on the gameplay side of HTP for a couple months now.

Warning: This is going to be a fairly technical post.

On the positive side, the FDG engine has been improved by an order of magnitude. It's gone from being naively simulated on the CPU, to simulated on the CPU using a Barnes-Hut octree solution with constraint relaxation used to bring nodes together as opposed to the previous spring solution (which was much more prone to 'exploding' during low frame rate conditions). Finally I just migrated the whole simulation to the GPU using a compute shader and reverted to the brute force approach, which, it turns out, works fantastically with massively multi-threaded capability of a GPU. In short, I can simulated ~100 fold more nodes than before:

So that's one of the changes to the core tech. There are 2 significant others, and they were also partially born out of this start-up endeavor. First, till recently, all game data that was procedurally generated was just stored in memory in a whole pile of different specialized classes, leaving me with this significant challenge of figuring out how to serialize it all to disk without losing any state. Then having to deserialize all that data and reinitialize any unsaved data to just the right state for things to carry on as expected. Hard given the scale of the simulation going on in HTP.

Then it kind of re-dawned on me, something I had discounted early on, because it didn't feel 'right' for a game. Just use a big ass NoSQL database. It's scheme-less document nature made it perfect for storing a shit-ton of different kinds of entities without spending a bunch of time developing a strict schema that would be rigid and resistant to future changes. It initially felt wrong to have a full blown database server running locally behind a game. But HTP has grown in scope to such an extent that it's not really your run-of-the-mill game any more.

The benefits of this change is huge, and it was driven by the need for a FDG engine agnostic to the dataset it is presenting. First of all, saving and loading are completely free. I simply don't have to worry about serialization or deserialization at all.

Everything in the world is an instance of the Entity class, which is supported by a backing document (in this case a MongoDB document). The class overloads the index operator [], and all data access for each entity is done by indexing it like a string object dictionary (plus a few convenience properties for things like name, type, etc). All these entities are kept in game memory, any time a change is made to any entity via the index operator, it is marked as 'dirty' and added to a set of dirty entities. A background thread which maintains a connection to the backing database pulls from this queue of dirty entities and updates/inserts them into the database. This results in a constant persistent version of the entire game state which is only ever a second or so old.

Gameplay wise, this is also interesting, because it completely removes the concept of saving and loading games, which is an aspect I've always been keen on for HTP. I want player actions to always carry weight, much like a rouge-like, though that's not quite what I'm trying to make with HTP. The idea that the game world and everything you do in it is a permanent action can be quite compelling.

Secondary benefits to this shift to a database backed game engine are equally significant. From a developers perspective, it allows me to create tools (which I've already done) that connect to the same local database and have a constant live perspective into everything about a running game's state. A potent tool for debugging and accelerating development. Secondly, and this is a big one, it could potentially (with a lot of design consideration) lead to free multiplayer, of a potentially MMO scale if designed carefully. With all game clients reading and writing to the same database.

So that's the first of the two major engine changes that have been recently, the second is equally significant and was again born out of the need for a dataset agnostic data visualization engine. It involves removing all the logic for the creation and updating of entities and their relationships from the core engine and moving them into a scripting engine, in this case, one of my favorites: Python (well, IronPython with its C# driver, but it's basically the same).

This separation of concerns allows the engine to remain lean and clean and delegate all of the game logic out into high level scripts, which can even be hot swapped at runtime, allowing for extremely rapid turn around times on code changes. It also means the final game will be extraordinary moddable, which could potentially be modded into entirely different games with no input from me. Part of the programmer in me desperately wants to open source this thing, but the other part is screaming "YOU'RE 2 MONTHS BEHIND ON YOUR RENT!", so closed it shall remain against my best wishes.

All this scriping is still a work in progress, but I've created a basic finite state machine (FSM) framework in Python that allows entities (so far I'm only really simulating people and mobile devices,  as this has been effectively a complete game rewrite) to inherit from base types, but override certain behaviors. For example, a CEO goes to bed at night just like anyone else, and heads to work like anyone else, but their behaviors at work are very much different than say, a programmer, or a janitor. So their behavior script can override the AtWork state accordingly. These scripts make it very easy to add new complex behaviors to entities, especially given the scheme-less nature of the backing database, as any script can introduce new data to an entity with easy. For instance, when I added mobile devices, it was just a few lines of script code to give them batteries that drain and are charged depending when their user plugs them in. Yes this is a potentially pointless piece of gameplay logic, but the easier it is to add all these small details, hopefully the easier it will be for players to discover emergent gameplay, which has always been my goal with HTP.

So for the TL;DR types: Development was sidetracked with good reason, which benefited the game engine significantly, and will hopefully continue to. Gameplay logic is being totally rewritten in Python script executed by a leaner, cleaner core game engine. Hopefully rapid development will ensue baring other work commitments.

As always:

Hacking a Much Bigger Planet

I've always found a silent dev blog is usually a good sign of continued development, and in this case that is very much the case. Development on HTP continues in earnest, and much progress has been made. I'm not going to try and covered everything that has changed since I last posted, there's simply too much to cover.

The game world has grown expenetially in recent months. This is partly due to the addition of "clusters" and "superclusters". Clusters can be thought of as cities, towns, villages, large campuses, military bases, etc. basically anything that would contain a number of individual networks which in turn contain a number of devices. Superclusters are collections of clusters, or put simply, countries.

Using carefully combined real world data, I've added every country in the world to the game (bar a few small island states with little or spurious data), and the largest ~4,500 cities in the world. The scale of a supercluster is determine using a combination of the countries total population and its GDP (which I currently use as a rough indicator of network density). Though I have also collected data on every country's internet penetration and total internet users, which I will find some use for I'm sure. The benefit of generating the world like this is that it allows me to defer the generation of actual networks until the player actually visits the cluster they are contained in. This gives me an effective order of magnitude increase in generation speeds.

While I only know the 'estimated' total world population of networks, devices, and users at generation-time, in practice, the player will not notice their absence before they are generated. Of course, to give the world some life and activity outside of the cluster the player begins in, I've selected a number of organizations (more on these later) in the world at generation-time and fully generated them and their employees (again, more on these later). This gives the world the illusion of being full of life and activity before the player visits more of its clusters on their travels.

Speaking of travels, a lot of work has gone into how this mechanic works. The player can only navigation around the world at a certain rate, which is plenty fast enough for navigating netween the networks in a cluster, but the distances between clusters is very large, and the distance between superclusters is vast. The 'connections' between clusters now act as a sort of highway, when the player approachs a connection, anywhere along its length, they are 'sucked' in and accelerating to great speed, pulling them to the distant connected cluster or supercluster. They can of course pull themselves out of the connection at any point, and I've got some plans for gameplay reasons for them to do so coming in the future. The current graphics for this are placeholder, but I have been playing around with some effects which I hope to incorporate soon:

Another benefit of travel via connections comes in world generation. The time spent travelling down a connection to a previously unvisited clusters gives me the time to generate that cluster before the player arrives, while the result still needs some graphics work, the effect will basically allow for seemless travel around a world in constant generation.

Dividing the game world up into clusters and superclusters will also allow me to scale the AI simulation around the world based on where the player is currently located. For instance, the player really doesn't need to know or care that Phil Robonson arrived 13 minutes late for work this morning and thus received a warning from his boss (more on this later), or that Sarah Peters learnt that her best friend is in serious debt (keep reading). But they certainly will care that the network administrator of the network they are in the middle of hacking just got paged after you triggered an alarm on the network's external firewall and is logging in to track you down. I still have to keep track of all the little details around the world, but most of them aren't that imporant from moment to moment and so can be batch simulated when they become relevant to the player. The player doesn't really care WHEN Robert Johnson from accounting got fired for fighting with one of his colleagues, they just need to know it happened if he becomes important to them later down the line.

As you might have realized, the simulation of the world has become quite a lot more detailed since I last wrote. HTP has taken an interesting turn in terms of gameplay, one I hope will further make it stand out in the small genre of hacking games. The level of detail in the simulation of the inhabitants of the world has been greatly increased. The anecdotes in the previous paragraph weren't just some hypothetical examples, they are all fully implemented and have real consequences to the player and all of the world's AI entities. As briefly mentioned earlier, I've added organizations to the game. Previously every network existed as an island of sorts, with no kind of controlling entity or real relationships to other entities (By 'entity' I mean anything tangible that interacts with other entities in the world. Organizations, networks, devices, people, etc.). Now when generated, networks are empty sites that are then allocated to an organization. Organizations vary in size, they can be small independent companies controlling a single network up to huge multinationals controlling hundreds of networks across the world.

When an organization sets up shop in a network they will procede to hire any employees they need to do what they do, this process involves looking at each position that needs to be filling in the network depending on what the organization does: administrators, programmers, bank managers, lawyers, managers, a CEO, etc. For each position, people will be randomly generated into the world, and in a kind of interview process, eventually someone with the appropriate personality traits will be selected (managers for instance, are required to be more demanding of their subordinates, and programmers are required to be of a certain level of intelligence). Once a suitable candidate has been found for the position, they are hired into the organization and are allocated to the network they will work in (not given a computer yet mind you, this is all done at cluster-level generation, not network-level, where devices are generated).

So now we have a bunch of employees working specific jobs at a specific place for a specific organization, we're half way there. These people don't just live to work, they must have lives outside of work as well. The next step is to find them a place to live. The generation engine next finds a home network somewhere nearby where they work and allocates them as a resident of said network. What about human relationships? Some people are single, some have partners, some are married, some are straight, some are gay, some are involved with people at work, some with people outside of work. All this is generated, so we end the cluster-level generation with thousands upon thousands of simulated people with their own personalities, lives and knowledge.

Knowledge. This is possibly the most important stage of generation. It's all well and good to have thousands of people going about their lives, but information is power, and information is the currency of HTP, and what information is there is no one knows anything? These most interesting part of developing this procedurally generated world for me has been simulating the accumulation and exchange of knowledge by the inhabiting entities. This process might be best described with a story. So I'm going generate a world right now with a random seed of 1632 (because it's 4:32pm currently), find a randomly selected person living in Vancouver, Canada (because that's where I live), let the world simulate for 1 month, and tell you their story.

• Our story begins with a 24 year old straight female named Jean Vaughn.
• She works at Central Telecommunication as an account manager.
• She lives at West Summit Plaza at 168 Hickory Crescent.
• She is in a relationship with Sean Brock, a 34 year old unemployed man.
• Jean's mobile number is (+1)681-6921.
• Jean's work computer's login is jean:jack.
• Jean is good at her job (with skill of 0.81), very ambitious (0.97), innoffensive (0.17), uncharismatic (0.16), loyal (0.74) and immoral (0.1), this will affect her actions.
• She is friends with her boss Megan Landry, a charismatic but somewhat confrontational 31 year old woman.
• Jean was recently given a verbal warning for being 18 minutes late for work.
• 58 other people live in the same apartment block as Jean.
• She lives on the 2nd floor, as well as 16 other people.
• Living next door to her is a 28 year old man named Raul Howe.
• He works at a Colocation run by North Telecommunication Co.
• North Telecommunication Co. is a small independent telecomms provider based out of Vancouver, Canada with 58 employees.
• While small the are very efficient and their last month's revenue was $3.3 million.
• Raul's work computer's password is 'ladybug'.
• It turns out Raul is working undercover for another organization, a secret he's so far kept to himself.
• Speaking of secrets, it also appears at some point in her past, Jean Vaughn hired a hitman.

I could go on, or let some more time pass and watch Jean and her friends' lives change as the world carries on, but I think this gives an idea of what's going on in the world at this stage. As a side note, the simulation of this world for a month took approximately 10 seconds.

What has made the development of this much detail possible is a tool I've been working on for the past 2 or so months. It's a standalone simulator of the game world that makes it much easier to see the vast amount of activity and information that the world simulation generates without all the overhead and obfuscation that the game engine itself creates. I've pulled all of the game logic out into a separate DLL that will let me run the game itself and the simulator on exactly the same code.

I sometimes wonder if spending so much time away from the game itself to create this simulator is worth it, then I look at how far the world simulation has come in the past 2 months, and how hard it would have been to make the same progress if limited by the game engine. I feel it has been completely worth it, and has certainly made it an order of magnitude easier to find and fix hard to spot bugs, as well as create the extremely high level of detail that will create a believable and interesting game world.

I hope this brings things back up to date, I'm sure I've not mentioned a ton of stuff I've added since April, but this is enough of a wall of text already and I don't want to overdo it.

As always... Hack the Planet!

Still Hacking

Well so much for keeping up with the blog. Every spare hour I've had over the past 2 months I've been spending on development so writing about it has taken been taking quite the back seat. I honestly don't know how indie developers find the time to keep dev blogs up to date. Well in news, I've had a chat with my boss about my day job, and I'm going to be cutting back from full-time to part-time, something almost unheard of in the games industry in my experience. This should give me not only at least another 20 hours a week to spend on HTP, but hopefully more time to keep this blog up to date and even, heaven horbid, have some free time!

HTP has come a long way since I last wrote, not only in terms of gameplay, but in visuals. I've replaced the previous placeholder UI with a 3D UI that whilst more or less functions the same way, looks much better. I always try to tell myself that graphics aren't that important and can just be added last, but it's hard to keep your morale up when all the other games you play look so much better than your own. I probably spend about a month completely redoing the UI engine and even though it has hasn't changed the gameplay much, it just *plays* better now. It also looks and feels more 'cyberspacey' than before.

(Please excuse the low quality/huge download size of the gifs, I'm new to this)

This isn't to say I haven't been working on gameplay at all. I've done a lot of work on the AI. Procedurally generated users now have lives, they go to work, they go out for lunch, they hang out in bars after work, and they go home to relax and sleep. They bring their mobile devices, connecting to networks with wifi as they move around, and have different routines depending where they are. If you, for instance, were to hack into a system administrator's workplace's sprinkler system while they were at work, and schedule a sprinkler test, they would be evacuated from the building and thus the network, leaving it more vulnerable to attack. You will even be able to get someone fired by planting illegal files on their machine and notifying the authorities. There is still much work to be done here, but the design is shaping up quite nicely to allow for some very interesting emergent gameplay.

I'm pretty happy with the visuals and UI at the moment, and I don't think I'll need to do much more work there for a while, leaving me free to get my teeth into the really juicy gameplay and AI behaviours. This is good, since it's something I'm much better at, and since very shortly I'll have much more time to spend on it, things should move very quickly. Exciting times

(I love how the 3D UI benefits from the crash post-effects in such an awesome way)

Thats all for now, once I move to part-time work over the next week or two, I'm going to try and plan my more plentiful time and make sure to allow for some hopefully weekly time to document my progress here.

Hack the Planet!

Almost a Game

So a lot of time has past since my last update. But this time I'm not going to blame being busy at work, or Christmas, because I've in fact been working very hard on HTP. Every day for the past 98 days according to Github in fact. Look, I've got proof!

I've been carrying on implementing the tutorial, with a few interesting detours along the way. I've added so much in fact I don't know where to start. Maybe a quick run down on what the tutorial contains will help, so here it is so far:

Implemented:
- Intro
- Give the player their computer
- Port scan a neighbouring computer
- Identify the neighbouring computer
- Hack the neighbouring computer with the provided OS exploit
- Install the provided spam bot
- Go to a neighbouring home network
- Port scan and identify the router
- Reveal vulnerability database on the map
- Download a remote admin vulnerability for the router
- Start developing their own exploit from the vulnerability
- Speed up time until the development is complete
- Return to the neighbouring network and hack the router
- Scan the network to reveal computer and wifi access point
- Identify the switch and wifi access point
- Hack the switch
- Scan the network behind the switch
- Find the computer with the target user behind the switch
- Hack the computer using the OS exploit
- Reveal a software vendor on the map
- Purchase a key logger from the software vendor
- Install a key logger on the computer
- Identify the target phone
- Download a sensitive target file from the mobile using the provided OS remote copy file exploit
- Reveal a data exchange on the map
- Sell the sensitive file on the data exchange

Not yet implemented:
- Return to the keylogged computer and download the newly placed cryptocoin wallet
- Return to the home computer and open the wallet using the keylogged password
- Profit!

There will of course be much more to the gameplay then what is included here, but these will make up the basics, with more advanced gameplay being introduced throughout the storyline.

I've added externally accessible device content, or in other words, websites. These are device's contents that can be accessed without hacking into a network first, and are currently visible as cubes jutting out of the surface of networks.

There are 3 types of websites at the moment. Software vendors, where there player can purchase software. Data exchanges, where the player can buy and sell files (this includes all kinds of files at the moment, including illegally obtained files, but this will be replaced with blackmarket websites in the darknet eventually), and vulnerability databases, where the player can find publicly available vulnerabilities and develop them into usable exploits.

This brings me to the next feature, players can now create exploits from vulnerabilities. These exploits take some time to develop (time acceleration has also been added) and target specific versions of software. Different vulnerabilities have different effects on the target device, some will allow you to remotely copy a file, some will disable the device entirely, and some will grant you full access to the machine, or 'root' access as it's called.

I also completely rewrote the UI system to be totally data-driven, I did this over the Christmas holidays, it was quite boring, but needed to happen, and the sooner the better, as now adding now UI is as simple as whipping up a new JSON file along the lines of this:

{
"Elements":
[
{
"Name": "Divider",
"Layout": "=================================="
}
],
"Arrays":
[
{
"Name": "Cheats",
"Items": ["Complete All Missions", "Lotsa Money", "Complete All Development", "Toggle Instant Processes", "Skip Tutorial", "Unhack Target"],
"Layout": "{ITEM}",
"MenuNavigation": true,

"ActionsOnSubmit":
[
{
"Action": "RunCheat",
"Parameter": "{SELECTION}"
},
{
"Action": "GoBack"
}
]
}
],

"Layout": "Cheats\n{Divider}\n{Cheats}",

"Links":
[
{
"Keys": ["Escape", "Backspace"],
"Actions":
[
{
"Action": "GoBack"
}
]
}
]
}

I've also moved to Unity 5 pro, courtesy of a 90 day beta license gifted to me at a Unity 5 workshop that happened here in Vancouver in December, thanks guys!

Most of the first half of January was spent totally refactoring how software works. Before each type of program had its own source file which defined its behaviour. It worked but wasn't that flexible, now everything is, you guessed it, data-driven. The behaviours and development lifespan of a software product line are all defined in JSON files which look like the following:

{
"Name": "ScanX",
"Category": "Scanner",
"Description": "Brute force port scanner, not very stealthy but gets the job done.",
"Architecture": "x86",
"Tags": ["PortScanner"],

"StartingVersion": 1.0,
"VersionStep": 0.5,
"StartingLevel": 0,
"StartingCost": 50,
"LevelStep": 10,
"MaxReleases": 20,
"CostStep": 5,
"FormatString": "{NUMERIC:F1}",

"ExecutedActions":
[
{
"Action": "ScanPorts",
"MinSpeed": 5, "MaxSpeed": 2.5,
"MinNoise": 10, "MaxNoise": 10,
"MinRequiredLevel": 0, "MaxRequiredLevel": 5
}
]
}

You'll notice mention of 'level' a couple times in there. The game now has a global tech level which slowly increases over time, when the level increases enough, a software's developer will release a new version of it to the public, which can then be purchased (or otherwise 'aquired') and installed, reverse engineered, cracked, etc. I want to make the release and acquisition of things as dynamic as possible. There should be nothing to stop a hacker breaking into a software developer or vendor's network and stealing their latest, possibly even unreleased, software and selling it on the black market. Hackers don't sign NDAs right? Alternatively, if you could get your hands on an unreleased operating system and find a vulnerability in it, you could be sitting on a pretty juicy zero day exploit. But why stop there, maybe if you can create a stealthy enough virus, you could infect said unreleased OS and find yourself will a botnet of thousands created for you. It's that kind of lateral thinking and emergent gameplay I want to make possible and encourage. I want to provide the tools and let the players create their own fun.

My initial goal was to complete the tutorial and get HTP to the stage I can honestly call it a game by my birthday later this month. I'm actually a little ahead of schedule on the tutorial and it should be finished in the next couple days. But HTP doesn't quite feel like a game yet, it's still quite stagnent and static, so I think I'm going to keep myself on my toes and revise that goal to include added some AI to the game. I want to see freelance whitehat hackers aquiring software, finding vulnerabilities in it, and releases those vulnerabilities to the public in vulnerability databases. I want to see blackhat hackers hacking into networks and devices in search of cryptocoin wallets, installing spam and keyloggers, and generally causing mischief. I also want to see average users just going about their life, spending their day at work, and their evenings at home, playing games, shopping online, downloading porn, etc.

I'll leave you with a random image to add a bit of colour. I'll try to update this more frequently so I don't have to pour though my github commit logs to remember what I've actually done in the past 2 months!

Hack the Planet!

Tutorial-Driven Development

I've come up with a new approach to development. Previously I would just choose the next feature I though HTP needed to slowly drive it towards gameplay completion, and tackle that. It was working, but it was slow and haphazard, also it resulted in a lot of the little details that make a game complete getting missed out.

My new approach is something I'm called "tutorial-driven development", I suppose it's a bit like test-driven development for games in a way. I started by planning out the entire tutorial sequence a new player will be presented with on the start of a new game. A sequence that when completed will give a new player a basic handle on the gameplay. It was a little intimidating to realize how much work was still left to do before I could call HTP a 'game' of any description. But it also gave me an excellent to-do list of large and small features I needed for that to happen. I've just been slowly chewing my through those features.

The first step was to create a mission chain system, since the tutorial would be made up as a series of missions, which might share common source users and target devices. For those interested, a mission chain template looks a bit like this:

{
"Contacts":
[
{
"Name": "A Friend",
"Type": "Hacker",
"IsMale": "false"
}
],
"Targets":
[
{
"Name": "NeighbouringComputer",
"DeviceTypes": ["Computer"],
"Vicinity": "HomeNetwork"
},
{
"Name": "NeighbouringNetworkRouter",
"DeviceTypes": ["Router"],
"Vicinity": "NeighbouringNetwork",
"Origin": "HomeNetwork"
}
],
"Chain":
[
{
"Description": "Welcome to the grid {ACCEPTINGUSERNAME},\n\nYou don't know me, but you can trust me. I was once like you, back when I was just getting started.\n\nI guess I feel for you.. I know how hard it can be, at the beginning. I'm going to help you, cause a long time ago, someone helped me, so it feels like the right thing to do. Besides, I could use a little good karma right about now.\n\nReply to this message and we can get started.\n\n - {SOURCEUSERNAME}",
"ShortDescription": "Welcome",
"ResponseMessage": "Let's get started.\n\n - {ACCEPTINGUSERNAME}",

"Goal":
{
"Type": "Respond",
},
"Source": "A Friend",

"DelayUntilNextMission": 2
},

...

I won't get into too much detail explaining this, as I'm trying to keep this update more gameplay-oriented, but I'm sure the coders out there can see where I'm going with this. Unfortunately these mission chain templates, unlike many of the other templates, are getting quite clunky to edit by hand, so at some point I think I'm going to have to pull my thumb out and write an editor for all these templates. I'm mainly putting it off cause it's boring as hell.

I've completed the first 9 tutorial missions so far, and in the process have been forced to code lots of little tiny features and polish existing ones, it's been a great exercise.

Another thing I've been working on has been revamping the connections between nodes. Until recently they were just simple 1 pixel wide lines that flashed with network activity, I always wanted them to be more then that, so to that end I've replaced them with true cylindrical meshes. Now now have real dimensions, and will be able to contain things like neat traffic effects, or even the player, acting as kinds of highways.

Obviously a lot is still to be done with their visuals, but I've proven the performance hit of having thousands of true 3D connections instead of simple lines is negligible.

Well that's all for now, just a quick update. The next thing I will be working on is letting web servers has a role defined, which governs what kind of web content they will serve. Starting with a vulnerability database needed for the next tutorial mission.

Hack the Planet!

Inspiring Bugs

Sometimes graphics bugs can inspire :]

Thin Vertical Slice

It's not even been a month yet, I feel like I'm jumping the gun with another update. But I've got so much done over the past few weeks I feel like if I left it any longer it would result in my brain exploding into an endless wall of text no one would care to read.

I'm going to present this in more of a bullet point form, as there are a bunch of small and large things I want to document here, so in more of less chronological order since since my last update:

Users:

Users are tangible things now, they have names, skills, homes and workplaces. They come in a variety of types, some separated by income brackets, some by affiliation. Different kinds of users will generate different amounts of network traffic, and their files and network traffic have different values based on what kind of user they are. A hacker will generate gobs of network traffic of a fairly high value, but has an extremely high security skill rating, which will make it much harder to steal from them. A low income average user has a lower security rating and lower value files and traffic.

Next thing I want to do with users is give them presence, so they can either be at work or at home, and depending on their job and function, this will effect gameplay. If a extremely skilled security officer in a corporate network is giving you grief, simply find out where he lives, hack into his home network, and set off his security alarm, causing him to go home and getting him off your back.

Movement:

I always wanted to tie movement into the connections between networks. I still want to maintain the feeling of flying freely through cyberspace, so I've compromised and calculated the player's speed based on their proximity to network connections and how much bandwidth those connections have. The result will be that a player must more or less follow the connections between networks, and faster connections will act as highways between distant networks.

I've also replace the old camera controls with a new physics based camera. I think it feels much more satisfying to get around with these new controls. The camera accelerates, drifts, can spin and roll and interacts with the world more realistically. I also added a neat impact effect using the new post-effects (more on that later):

Traffic Visualization:

Network traffic is visible now on the connections between networks. Depending on that kind of devices are connected, and who is using those devices, different amounts of traffic is generated. A printer talking to a home network's switch will create very little traffic, whilst a web server connected to a firewall will create a significant amount of traffic. These effects can be seen in the video below, and are made all the more obvious with the new bloom effect (again, more on that later):

Network Entrances:

Networks are now contained spaces, with entrances consisting of devices like routers. You can 'pass through' a network by flying around it, but if you want to access the internal devices in the network, you first have to gain access to one of these entrances.

Once inside a network, they are kind of self-contained little levels, which you fly around and hack your way through. Still a lot of work to do here, I want to have the outside world sill visible through the boundaries of the network, though slightly blurred. And I would love to differ the style and appearance of varying types of networks. Cold, monolithic regimented looks for military and government networks, and warm, cozy, loose looks for home networks.

Post-Effects:

The Unity asset store takes most of the credit for these improvements. I found an excellent old TV tube shader to create the scanline, noise and chromatic aberration effects (check it out here: https://www.assetstore.unity3d.com/en/#!/content/24046). I also added some bloom, glow, vignette and FSAA effects to some cheap graphics wins. The improvement in appearance should tide me over for a while and make me less embarrassed to show off what I have so far.

Messaging System:

This was going to start out as a tutorial system for new players, but I think will become much more integral to gameplay in the future. It's basically an email inbox. When writing the initial introduction message, it became apparent that a storyline could very well fit into this game, and as much as I don't really care for storylines in games, I do kind of look forward to writing one, no matter how flimsy, for HTP.

Missions:

After creating a basic messaging system, I got a bit carried away and ended up writing a random mission generation system. I didn't really plan on having missions in HTP, but after having added them, I'm not sure how it would have ever worked without them. Here's a short video running through a simple 'steal the file' mission. The mission is only completable with debug tools, but it gets the idea across:

Name Generation:

Finally, after having added missions, and not wanting to simply waypoint the mission's goal out blatantly, I added random network name generation to the world. This will help me guide the player to a mission's target without directly revealing it, "Please steal file X from Mrs. Y, she lives near Z.". This was pretty fun to implement, I do love seeing procedural generation in the wild :) Though I have to say, most of the business names come off as a little 'new media' at the moment :S

Well that's it for now, I think I've got enough off my chest to carry on with the next push. I think the next thing on my plate is giving money a purpose and adding 'shops' to the game. Don't know how they're going to work yet, but I'll figure something out.

Now if you'll excuse me, there's 23 punk bands playing tonight for $15 and I need to get my drink on...

Hack the Planet!

Gameplay Brainstorming

Some lunch break gameplay ideas to help guide my development:

• A frontal attack is too difficult on a high security target, so instead, discover a nearby person who works at the target and infect their phone or laptop with malware that has a chance of spreading to the internals of the target network.
• If a target's primary entry point is too secure, but a currently unused backup entry point is less secure, a DOS or other disabling attack to the target's primary entry point could result in the backup entry point becoming active, allowing for entry into the network.
• Some extremely high security, high value networks are air-gapped with the exception of a highly controlled satellite connection via another high security network.
• A target's security personal are only human and can be distracted by attacks on secondary targets on a network, lessening the chance of detection of an attack on the primary target.
• Harnessing the power of a botnet can make it possible to crack crypto-coin wallets.
• Networks containing devices owned by VIPs can yield high value data which can be sold on the blackmarket.
• Messing with VIPs' data will attract the attention of the authorities, raising your suspicion level.
• A high suspicion level will cause the authorities security professionals to actively seek you out. You can reduce your suspicion and lower your chance of arrest by moving your home base.
• Dwelling at the same home base for too long with a high suspicion level will result in the authorities probing your network. If these probes go unnoticed for too long, you will be raided and arrested. Taking the appropriate precautions and looking out for activity by the authorities at your home base will keep you one step ahead.
• Moving your home base takes time and costs significant amounts of money. The new location should be chosen carefully.
• Software and exploits can be purchased from other hackers, finding these hackers can be difficult and some will not deal with you until you've reached a certain skill level.
• The presence of another hacker on a network can be identified by the unusual behavior of a device on the network.
• Other hackers have home bases as well, containing their collection of software and exploits, these devices are extremely high security and difficult to attack. Doing so will not raise your suspicion level with the authorities, but will likely result in retaliation if detected.
• Security advisories can be translated into exploits with a high enough skill level.
• Every workstation has a user. Every user has a home network their other devices may be found on while they're not at work.

I've also added filesystems, money and time too, so here's a picture to keep things colorful:

5k lines of code lighter...

So another month has past, there seems to be a pattern forming here. Maybe I'll make this a monthly update kinda deal. I've been finding much more time to spend on HTP over this last month, mainly because I've been making it a high priority to find the time. Things have been going very well, my decision to throw out my old art style was definitely the right one, and actually led to me throwing almost my entire code base away, a depressing thing as any programmers out there will know, but almost always worth the pain.

I had a gameplay epiphany the other day, on the toilet of course, where all epiphanies must occur. Up till then, I was never really sure how I was going to translate this strange simulated internet thing I've been creating into some kind of game, something fun. Up till then, I was adamant on simulating every tiny little aspect of everything as a node in a hierarchy. The world had cities, cities had sites, sites had networks, networks had computers, and computers had components. It was kinda cool, being able to zoom right down from the world level to the CPU level. Cool, but extremely costly. To generate the 100,000+ computers I wanted to make the game feel 'vast' meant simulating literally millions and millions of nodes. All the time. It didn't scale, and was really leading me to any kinda of gameplay.

So on the toilet, it dawned on me, I could simulate thousands of nodes without much difficulty, so if these nodes weren't individual components, but entire networks, I could easily have the hundreds of thousands of computers I wanted, and each of these networks, which would form the galaxy of interconnected nodes floating in cyberspace, the experience I wanted to create, could be its own sort of 'level', with its own puzzles and challenges. It's kind of hard to articulate how I think this is going to work in my head, so I supposed it's probably easier to just show you. So here is the closest I've come to a thin vertical slice of how the finished game might play so far. You'll notice for the first time a few actual gameplay elements going on as I take on a small home network and infect a computer with some malware:

I'm much happier with the visuals, as primitive as they are, I think I could do a lot with them. It's also nice simulating networks like this because I can happily simulate the physics on 100-200+ computers in these networks and get that nice movement as new nodes are discovered and the network fleshes out.

In terms of the user interface, I started out trying to use a 3D interface that surrounded each node and displayed relevant information, but I found it hard to read, clunky, and getting in the way of the cyberspace feel. The current system console UI started out as a quick and dirty debug placeholder, but it's starting to grow on me, I think with some love, it could actually work for the final UI. It suits the hacker motif and I can convey a lot more information then I could with a 3D in-world UI. So it'll do for now, and I'll see how it pans out later as the game gets more complex.

Whilst the gameplay is still very very early on, I'm still trying to be very careful to try and not corner myself into repetitive gameplay, so am trying to think of ways to encourage lateral thinking and emergent puzzles. For instance, when attempting to hack an ISP or other high security network, a full frontal attack may be extremely difficult, with layers of firewalls and intruder detection systems. But ISPs must have a human element, an alternative approach would be to identify a nearby home network in which one of the ISP's employees works, then attempt to hack through their relatively low security home network and plant some malware on their mobile phone whilst it's connected to their wifi. This malware may in turn find it's way onto the internal network of the ISP and provide you with a backdoor that would be otherwise very hard to establish. It's this kind of gameplay I want to encourage, and while I'm a long away from that level of gameplay now, it helps to keep that in the back of my head at all times.

Well I guess that'll do for this month, I'm very excited with where HTP is at now, mostly because I've gotten over a lot of the technical challenges now, and am able to work on some of the pure gameplay elements.. which is not only more fun, but much easier work in comparison. The next month should be quite productive if I can still make the time to work on HTP.

Hack the Planet!

Bloody Crunch

It's been a long time since I both had a chance to write something on here or even work on HTP. The promised 2 weeks of The Crunch turned into 4 weeks, as always seems to be the way. The Crunch simply prevented me spending any time at all on HTP. As I'm sure some of the other developers out there know, time spent away from a project slowly builds up a barrier to re-entry, as knowledge of the code base fades and momentum is lost.

Crunch time is starting to end at my work, and for my own sanity I need to take my personal project time more seriously and make time for it.

This is not going to be a technical blog post for once, this is I guess more my ramblings about how, as developers, we tackle code block (the equivalent of writer's block for the non-programmers out there) and keeping project momentum going. I've been in this situation many times, and I've slowly learnt how to deal with it.

When approaching a project I haven't touched in a little while I don't try and pickup where I left off with the bugs I was tackling those many weeks ago. There's no point, my train of thought has long since left the station and you need those first steps back into a project to be fun and exciting.

So on that note, I've going to throw away my whole visual style and try some experiments. As much as I like the idea of flying around a globe with with real geographically accurate locations, the technical challenges are substantial, and I kind of feel I'll be losing the whole otherworldly feeling of drifting through the alien world of cyberspace. By limiting myself to a flat surface (which obviously the world isn't, I'm no flatlander.. but for all intents and purposes it is when you're zoomed in looking at a individual computer), I feel like I might lose some of the experience.

I plan on turfing out my entire graphics layer and trying something more akin to flying through a galaxy of connected nodes, to do this with my current code base would no doubt require MASSIVE refactoring, and would likely kill my attempt to regain the momentum I lost after taking a break from the project. So I'll be playing around with a few standalone demos probably, seeing how each one feels and what looks right. I'll post my results on here and whilst I know I don't have many followers on here, I'd love to hear feedback and opinions when I do :]

I'll leave you with a reference picture I've had kicking around for a while which I always find particularly inspiring:

Hack the Planet!

It's STILL alive!

It's been exactly a month since my last update, my excuse is a lot of work commitments and The Crunch have really sapped my time. But 2 more weeks of The Crunch and I should have more time to spend on HTP.

Having said that, I have managed to squeeze in the odd hour or two here and there and I've actually made some good progress on something I'd like to share with the tumbleweeds :]

I've completely rewritten how the signal routing system works and have finally got it up and running. Nodes can now now 'provide' or 'request' things based on their functions, for example, here is part of the definition of a computer's power supply:

"Functions":
[
  {
    "Type": "Request",
    "ResourceType": "MediumVoltagePower",
    "Interval": 5,
    "ShowRequest": true,
    "Critical": true
  },
  {
    "Type": "Provide",
    "ResourceType": "LowVoltagePower",
    "Channel": "Power",
    "MaxPerSecond": 100
  }
]

What this means is every 5 seconds a power supply will attempt to request a unit of 'MediumVoltagePower'. Initially a power supply has no idea where it can get this request fulfilled, so it sends a broadcast request to every other node it's connected to. Put simply, this broadcast request will spread out across the entire power network until it encounters a node which has a 'provide' function that can fulfill the request. When this happens, the providing node sends a signal back down the reverse route that the requesting signal took. When the requesting node receives this signal, it now knows of a node that can provide for future requests so it stores the route to that node for future use.

I do a few other things to help speed this broadcasting process up, like if a request signal passes through a node that can't provide for the request, but does know of a node that can, it will point the request in the right direction, speeding the process up. Also, if any node is asked to pass on a providing signal, it will know where that signal came from, so it knows if it, or any other requesting node, needs that particular resource, it knows where to find it.

Well that's the dry details of how the system works out of the way. What's quite interesting about this system is pretty much all the traffic on the internet can be boiled down to requests and provisions. Some of them are critical for a node's operation, like power. But some are simply the byproducts of the behaviors of the AI agents in charge of the nodes. Take the following example for taken from the definition of the 'AveragePersonComputer' role (roles are something I use to describe the behaviors of a node):

"Functions":
[
  {
    "Type": "Request",
    "ResourceType": "Network",
    "Channel": "SocialNetworking",
    "TimesPerDay": 20,
    "ShowRequest": true,
    "Critical": false
  },
  {
    "Type": "Request",
    "ResourceType": "Network",
    "Channel": "Banking",
    "TimesPerDay": 2,
    "ShowRequest": true,
    "Critical": false
  },
  {
    "Type": "Request",
    "ResourceType": "Network",
    "Channel": "Porn",
    "TimesPerDay": 1,
    "ShowRequest": true,
    "Critical": false
  },
  {
    "Type": "Request",
    "ResourceType": "Network",
    "Channel": "Piracy",
    "TimesPerDay": 1,
    "ShowRequest": true,
    "Critical": false
  }
]

None of these requests are vital for the operation of the computer, but they will be requested by the node as long as it's active. These behaviors might change over time too, if a computer is infected by malware and becomes part of a botnet, it may start producing spam traffic as well as it's normal traffic.

This is all kind of exciting for me, cause the signal routing system is one of the first big systems I need to start doing some interesting stuff, stuff even bordering on gameplay! :O It's also really exciting to see the world come to life for the first time:

Having spent a good chunk of my day today trying to tear out quite an impressive infestation of spyware, browser bars, search hijackers and all kinds of other nasty computer bed bugs from my girlfriend's laptop, I can't wait to start simulating that kind of stuff in my little internet :]

Hack the Planet!

Home Sweet Home

Wanted to just quickly share a screenshot (click to expand):

This is someone's home network. You can see the external network and power connections on the border there, they connect to the outside world and relay network and power signals as needed. The fuse box, for lack of a better name, feeds off this power, and in turn powers all the wired devices in the house, indicated by the yellow wires. The router is connected to the outside world and serves the house's 3 computers (it turns out this is quite a large house, 3 is the current maximum for a home network) and their wifi access point, indicated by the green wires. The access point appears to have a mobile phone and laptop connected to it wirelessly, indicated by the white lines.

I just found this screenshot quite satisfying, the procedural generation is coming along quite well, and it's nice to see it working after this huge refactoring I've been doing. For those interested, below is the template file for home networks:

{
  "ShouldScale": true,
  "MinSize": 0.05,
  "MaxSize": 0.1,
  "Needs": ["MediumVoltagePower", "Network"],
  "Structure":
  [
    {
      "Type": "FuseBox",
    },
    {
      "Type": "Router",
      "Serves":
      [
        {
          "Type": "Computer",
          "Min": 1,
          "Max": 3
        },
      ]
    },
    {
      "Type": "WifiAccessPoint",
      "Serves":
      [
        {
          "Type": "Mobile",
          "Min": 0,
          "Max": 3,
          "Hours": ["Morning", "Evening", "LazyNights"]
        },
        {
          "Type": "Laptop",
          "Min": 0,
          "Max": 3
        }
      ]
    }
  ]
}

The Whole World at Once

So I'm about half-way through a major refactoring of the procedural generation system, it's coming along slowly. It's one of those refactorings when your code base seems to half in size. I usually take that as a good sign.

Up till now the world wasn't generated until it became 'relevant'. This means until you zoomed in on a city, it was nothing but a placeholder. When it became relevant, it would generate all it's component parts: power stations, ISPs, neighborhoods, etc. The problem with this though, was that a freshly generated world had no knowledge of its constituent parts, so was unable to generate network traffic on their behalf. This would have led to a very boring world.

The brute force solution would have been to just generate the entire world at loading, the problem with this though is the majority of the CPU work done during generation is related to the physical representation of a node. All we really need to know about a node beforehand is what it is, how big it is, and what it belongs to. So what I've been refactoring towards, is a system where the entire internet is generated at the start, but only at a minimal level. This meant breaking apart the relationship of a node's details and its graphical representation.

I'm nearly done this refactoring now, and it's been interesting to see some of the numbers it's producing. Up till now I've not really had any real idea how big the internet I've been creating is, since it would only generate nodes when I looked at them. But now since I'm generating all of it beforehand, at least at a minimal level, I know exactly how big it is.

So far, for the sake of fast turnaround, I've only been generating 200 cities in the world. It's been enough to get a fairly populated internet and takes about 20 seconds to generate (with completely unoptimized code mind you). So for the first time I've had a chance to see what my world is made of, here it is:

51605 CPUs
51605 PSUs
51605 OSs
51605 NetworkAdapters
6248 Routers
6008 WifiAccessPoints
6008 HomeNetworks
1041 SmallPowerSubstations
852 NetworkSubstations
852 Neighbourhoods
189 LargePowerSubstations
173 NetworkProviders
200 Locations
208 NetworkExchanges
16 Generators
16 PowerStations
16 ISPs

This translates to roughly 50,000 devices. Some of them desktop machines, some laptops, some phones, some routers, anything with a CPU. I'm pretty happy with this number considering it only takes 20 seconds to generate with the current unoptimized code. I'd ideally like to see upwards of a million devices in the finished game, but I'm actually less worried about the generation time and more about the graphics load and amount of UI clutter.

I plan to dump Unity at some point in the future and roll my own engine for this. I'm using very little of the goodies and niceties that Unity provides for most types of games, and it doesn't give me the low level rendering support I need to achieve the framerate I want. For instance, it doesn't provide access to GL vertex buffers so I'm pushing the thousands upon thousands of lines to the GPU every frame. With 1000+ cities in my world, the framerate drops below 15 fps on my little tablet. With vertex buffers I suspect this will rise to 60+ fps. But for now I'm putting off this migration, functionality first, optimization second and all that jazz.

I started this post with the full intention of talking about potential gameplay ideas and lay off the technical stuff, but I guess my fingers had a different idea. Well, it's always nice to get this technical stuff off my chest. Maybe after a couple more beers I'll rant about botnets and cryptocurrencies for a while.

Hack the planet!

It's Alive!

I haven't posted in a little while because I've been waiting for the next natural breaking point in development, where I have something significant to say.

Making things like this always feels a bit like a long road trip to me, you've got your mountains, your valleys, your long stretches. Right now I feel a bit like I've finally reached the ridge after a long uphill stint. Suddenly a bunch of things have come together and started working and there's a great sense of possibilities.

The things in this simulated world have finally started to talk to each other. I wanted to start with something simple, so I decided to get the power grid working, starting with the power stations. The first official breath of life in my world started in the power station located in Jakarta (which happens to be positioned almost directly beneath the camera's initial position, so it gets a lot of attention). I zoomed in to it until I could see it's constituent parts. The "SuperHighVoltagePower" generator (I really should find out the technical terms for this stuff) sputtered into life after it's initial 5 second delay period. It's produced a signal of "1" on the SuperHighVoltagePower wire which slowly traveled down it's connected wires, leading it to the power station's external power connection. The connection, as defined by its relay function (described in my last post), transmitted this power signal over to the adjacent Large Power Substation, who's job it is to bring the voltage of the power down from SuperHighVoltagePower to HighVoltagePower (yeah... I'm a scientist me) and transmit it to the other power-needy sites in Jakarta.

What I quite like about what I've created so far is all these behaviors are pretty much all defined in these easily modifiable JSON files. The only behaviors that live in code so far are the functions' logic, and given how very atomic and simple they tend to be, I will likely pull that behavior out into a scripting language, probably LUA, to make sure they are moddable too.

The next interesting challenge is going to be getting a city to adopt the behaviors of all its children and its children's children, so looking at a freshly generated city, you would see it spitting out the network traffic and power output you would expect to see it's as-of-yet ungenerated children produce. I could brute force this using a recursive approach that scoops up any important functions of a city's children, and run those functions on the children's behalf until they have been generated and can take over. But I think I'd prefer to start work on a more scallable kinda "traffic profile" system.

I'd want the traffic, be it network and power, to be produced by a traffic profile that is defined by the the profiles of a node's children. Instead of running hundreds of individual functions on the children's behalf, it would lump all this behavior together into a single traffic profile. If changes were to happen to this profile, say the city receives a number of new virus infections, when the player zooms in on the city and its children become relevant again or are generated for the first time, these new infections would be distributed to the node's children, and when zooming out again, the profiles would be consolidated back into its parent's profile. This feels like the only way to simulate an approximation of the entire internet on a little tablet :S

So that will likely be taking up quite a few of my next lunch breaks. Will report back when I've made some progress there :]

Keeping it Moddable

I've always been a huge fan of heavily moddable games, and I derive a bit too much pleasure from creating purely data-driven game systems. Not only for the benefit for the end user, but for my own sanity, I like fast turn around times and quick iterations, and if that means editing a simple JSON file vs recompiling the source I'll happily put the effort in to make things work this way.

Up till now, everything generated in HTP has been defined by node templates, as shown briefly in my last post. But till now, these node templates have only defined the structural qualities of a node. The size, name, requirements, provisions and children. Not the functionality.

In keeping with the data-driven approach, I want to keep as much of the functionality of these nodes in their template files as well. So I've introduced the concept of 'functions' to the templates. As an example, here is the new CPU template:

{
   "ShouldScale": false,
   "StaticNodes": true,
   "MinSize": 0.1,
   "MaxSize": 0.1,
   "Needs": ["Power", "Network", "Storage"],
   "Provides": ["Cycles"],
   "MinimumNeeds": ["Power"],
   "Functions":
   [
      {
         "Type": "ProvideResource",
         "ResourceType": "Cycles",
         "Amount": 1,
         "Interval": 5
      },
      {
         "Type": "Relay",
         "SignalTypes": ["Network", "Storage"]
      }
   ]
}

Here you can see that a CPU node offers two functions. First, when it is powered up and active (determined by the new 'MinimumNeeds' property), it will produce 1 'cycle' every 5 seconds. These cycles will be carried down bus connections to other nodes, notably the OS node, which will power up and start providing its functionality. The second function of the CPU is it to act as a middleman and relay certain types of signals from other nodes to all connected nodes, in this case network and storage signals to the network adapter and hard drive respectively.

It's still early days with this new system, but the goal to keep as much functionality in the data files as possible, to the point where I might look into replacing these function definitions with some kind of scripting language in the future.

In other news, I picked up a copy of Light the other day, not because it's my kind of game especially, though it is very good. But mainly because some of the visuals are exactly the kind of thing I've been imagining for HTP. The glitchy, staticy menu effects are fantastic, I've actually emailed the developer hoping for some friendly shader advice if they don't mind giving it.