Hack The Planet
Sunday 27 March 2016
Beneficial Distractions
OK, so I have a legit excuse for my posting absence this time. It turns out the force directed graph (FDG) engine I have been developing to power the graphics of HTP has drawn some interest from the data visualization market. So myself and couple friends are spinning up a start-up company around some of this tech. It's exciting stuff, but has also torn my away from working on the gameplay side of HTP for a couple months now.
Warning: This is going to be a fairly technical post.
On the positive side, the FDG engine has been improved by an order of magnitude. It's gone from being naively simulated on the CPU, to simulated on the CPU using a Barnes-Hut octree solution with constraint relaxation used to bring nodes together as opposed to the previous spring solution (which was much more prone to 'exploding' during low frame rate conditions). Finally I just migrated the whole simulation to the GPU using a compute shader and reverted to the brute force approach, which, it turns out, works fantastically with massively multi-threaded capability of a GPU. In short, I can simulated ~100 fold more nodes than before:
So that's one of the changes to the core tech. There are 2 significant others, and they were also partially born out of this start-up endeavor. First, till recently, all game data that was procedurally generated was just stored in memory in a whole pile of different specialized classes, leaving me with this significant challenge of figuring out how to serialize it all to disk without losing any state. Then having to deserialize all that data and reinitialize any unsaved data to just the right state for things to carry on as expected. Hard given the scale of the simulation going on in HTP.
Then it kind of re-dawned on me, something I had discounted early on, because it didn't feel 'right' for a game. Just use a big ass NoSQL database. It's scheme-less document nature made it perfect for storing a shit-ton of different kinds of entities without spending a bunch of time developing a strict schema that would be rigid and resistant to future changes. It initially felt wrong to have a full blown database server running locally behind a game. But HTP has grown in scope to such an extent that it's not really your run-of-the-mill game any more.
The benefits of this change is huge, and it was driven by the need for a FDG engine agnostic to the dataset it is presenting. First of all, saving and loading are completely free. I simply don't have to worry about serialization or deserialization at all.
Everything in the world is an instance of the Entity class, which is supported by a backing document (in this case a MongoDB document). The class overloads the index operator [], and all data access for each entity is done by indexing it like a string object dictionary (plus a few convenience properties for things like name, type, etc). All these entities are kept in game memory, any time a change is made to any entity via the index operator, it is marked as 'dirty' and added to a set of dirty entities. A background thread which maintains a connection to the backing database pulls from this queue of dirty entities and updates/inserts them into the database. This results in a constant persistent version of the entire game state which is only ever a second or so old.
Gameplay wise, this is also interesting, because it completely removes the concept of saving and loading games, which is an aspect I've always been keen on for HTP. I want player actions to always carry weight, much like a rouge-like, though that's not quite what I'm trying to make with HTP. The idea that the game world and everything you do in it is a permanent action can be quite compelling.
Secondary benefits to this shift to a database backed game engine are equally significant. From a developers perspective, it allows me to create tools (which I've already done) that connect to the same local database and have a constant live perspective into everything about a running game's state. A potent tool for debugging and accelerating development. Secondly, and this is a big one, it could potentially (with a lot of design consideration) lead to free multiplayer, of a potentially MMO scale if designed carefully. With all game clients reading and writing to the same database.
So that's the first of the two major engine changes that have been recently, the second is equally significant and was again born out of the need for a dataset agnostic data visualization engine. It involves removing all the logic for the creation and updating of entities and their relationships from the core engine and moving them into a scripting engine, in this case, one of my favorites: Python (well, IronPython with its C# driver, but it's basically the same).
This separation of concerns allows the engine to remain lean and clean and delegate all of the game logic out into high level scripts, which can even be hot swapped at runtime, allowing for extremely rapid turn around times on code changes. It also means the final game will be extraordinary moddable, which could potentially be modded into entirely different games with no input from me. Part of the programmer in me desperately wants to open source this thing, but the other part is screaming "YOU'RE 2 MONTHS BEHIND ON YOUR RENT!", so closed it shall remain against my best wishes.
All this scriping is still a work in progress, but I've created a basic finite state machine (FSM) framework in Python that allows entities (so far I'm only really simulating people and mobile devices, as this has been effectively a complete game rewrite) to inherit from base types, but override certain behaviors. For example, a CEO goes to bed at night just like anyone else, and heads to work like anyone else, but their behaviors at work are very much different than say, a programmer, or a janitor. So their behavior script can override the AtWork state accordingly. These scripts make it very easy to add new complex behaviors to entities, especially given the scheme-less nature of the backing database, as any script can introduce new data to an entity with easy. For instance, when I added mobile devices, it was just a few lines of script code to give them batteries that drain and are charged depending when their user plugs them in. Yes this is a potentially pointless piece of gameplay logic, but the easier it is to add all these small details, hopefully the easier it will be for players to discover emergent gameplay, which has always been my goal with HTP.
So for the TL;DR types: Development was sidetracked with good reason, which benefited the game engine significantly, and will hopefully continue to. Gameplay logic is being totally rewritten in Python script executed by a leaner, cleaner core game engine. Hopefully rapid development will ensue baring other work commitments.
As always:
Tuesday 24 November 2015
Hacking a Much Bigger Planet
I've always found a silent dev blog is usually a good sign of continued development, and in this case that is very much the case. Development on HTP continues in earnest, and much progress has been made. I'm not going to try and covered everything that has changed since I last posted, there's simply too much to cover.
The game world has grown expenetially in recent months. This is partly due to the addition of "clusters" and "superclusters". Clusters can be thought of as cities, towns, villages, large campuses, military bases, etc. basically anything that would contain a number of individual networks which in turn contain a number of devices. Superclusters are collections of clusters, or put simply, countries.
Using carefully combined real world data, I've added every country in the world to the game (bar a few small island states with little or spurious data), and the largest ~4,500 cities in the world. The scale of a supercluster is determine using a combination of the countries total population and its GDP (which I currently use as a rough indicator of network density). Though I have also collected data on every country's internet penetration and total internet users, which I will find some use for I'm sure. The benefit of generating the world like this is that it allows me to defer the generation of actual networks until the player actually visits the cluster they are contained in. This gives me an effective order of magnitude increase in generation speeds.
While I only know the 'estimated' total world population of networks, devices, and users at generation-time, in practice, the player will not notice their absence before they are generated. Of course, to give the world some life and activity outside of the cluster the player begins in, I've selected a number of organizations (more on these later) in the world at generation-time and fully generated them and their employees (again, more on these later). This gives the world the illusion of being full of life and activity before the player visits more of its clusters on their travels.
Speaking of travels, a lot of work has gone into how this mechanic works. The player can only navigation around the world at a certain rate, which is plenty fast enough for navigating netween the networks in a cluster, but the distances between clusters is very large, and the distance between superclusters is vast. The 'connections' between clusters now act as a sort of highway, when the player approachs a connection, anywhere along its length, they are 'sucked' in and accelerating to great speed, pulling them to the distant connected cluster or supercluster. They can of course pull themselves out of the connection at any point, and I've got some plans for gameplay reasons for them to do so coming in the future. The current graphics for this are placeholder, but I have been playing around with some effects which I hope to incorporate soon:
Another benefit of travel via connections comes in world generation. The time spent travelling down a connection to a previously unvisited clusters gives me the time to generate that cluster before the player arrives, while the result still needs some graphics work, the effect will basically allow for seemless travel around a world in constant generation.
Dividing the game world up into clusters and superclusters will also allow me to scale the AI simulation around the world based on where the player is currently located. For instance, the player really doesn't need to know or care that Phil Robonson arrived 13 minutes late for work this morning and thus received a warning from his boss (more on this later), or that Sarah Peters learnt that her best friend is in serious debt (keep reading). But they certainly will care that the network administrator of the network they are in the middle of hacking just got paged after you triggered an alarm on the network's external firewall and is logging in to track you down. I still have to keep track of all the little details around the world, but most of them aren't that imporant from moment to moment and so can be batch simulated when they become relevant to the player. The player doesn't really care WHEN Robert Johnson from accounting got fired for fighting with one of his colleagues, they just need to know it happened if he becomes important to them later down the line.
As you might have realized, the simulation of the world has become quite a lot more detailed since I last wrote. HTP has taken an interesting turn in terms of gameplay, one I hope will further make it stand out in the small genre of hacking games. The level of detail in the simulation of the inhabitants of the world has been greatly increased. The anecdotes in the previous paragraph weren't just some hypothetical examples, they are all fully implemented and have real consequences to the player and all of the world's AI entities. As briefly mentioned earlier, I've added organizations to the game. Previously every network existed as an island of sorts, with no kind of controlling entity or real relationships to other entities (By 'entity' I mean anything tangible that interacts with other entities in the world. Organizations, networks, devices, people, etc.). Now when generated, networks are empty sites that are then allocated to an organization. Organizations vary in size, they can be small independent companies controlling a single network up to huge multinationals controlling hundreds of networks across the world.
When an organization sets up shop in a network they will procede to hire any employees they need to do what they do, this process involves looking at each position that needs to be filling in the network depending on what the organization does: administrators, programmers, bank managers, lawyers, managers, a CEO, etc. For each position, people will be randomly generated into the world, and in a kind of interview process, eventually someone with the appropriate personality traits will be selected (managers for instance, are required to be more demanding of their subordinates, and programmers are required to be of a certain level of intelligence). Once a suitable candidate has been found for the position, they are hired into the organization and are allocated to the network they will work in (not given a computer yet mind you, this is all done at cluster-level generation, not network-level, where devices are generated).
So now we have a bunch of employees working specific jobs at a specific place for a specific organization, we're half way there. These people don't just live to work, they must have lives outside of work as well. The next step is to find them a place to live. The generation engine next finds a home network somewhere nearby where they work and allocates them as a resident of said network. What about human relationships? Some people are single, some have partners, some are married, some are straight, some are gay, some are involved with people at work, some with people outside of work. All this is generated, so we end the cluster-level generation with thousands upon thousands of simulated people with their own personalities, lives and knowledge.
Knowledge. This is possibly the most important stage of generation. It's all well and good to have thousands of people going about their lives, but information is power, and information is the currency of HTP, and what information is there is no one knows anything? These most interesting part of developing this procedurally generated world for me has been simulating the accumulation and exchange of knowledge by the inhabiting entities. This process might be best described with a story. So I'm going generate a world right now with a random seed of 1632 (because it's 4:32pm currently), find a randomly selected person living in Vancouver, Canada (because that's where I live), let the world simulate for 1 month, and tell you their story.
I could go on, or let some more time pass and watch Jean and her friends' lives change as the world carries on, but I think this gives an idea of what's going on in the world at this stage. As a side note, the simulation of this world for a month took approximately 10 seconds.
What has made the development of this much detail possible is a tool I've been working on for the past 2 or so months. It's a standalone simulator of the game world that makes it much easier to see the vast amount of activity and information that the world simulation generates without all the overhead and obfuscation that the game engine itself creates. I've pulled all of the game logic out into a separate DLL that will let me run the game itself and the simulator on exactly the same code.
I sometimes wonder if spending so much time away from the game itself to create this simulator is worth it, then I look at how far the world simulation has come in the past 2 months, and how hard it would have been to make the same progress if limited by the game engine. I feel it has been completely worth it, and has certainly made it an order of magnitude easier to find and fix hard to spot bugs, as well as create the extremely high level of detail that will create a believable and interesting game world.
I hope this brings things back up to date, I'm sure I've not mentioned a ton of stuff I've added since April, but this is enough of a wall of text already and I don't want to overdo it.
As always... Hack the Planet!
The game world has grown expenetially in recent months. This is partly due to the addition of "clusters" and "superclusters". Clusters can be thought of as cities, towns, villages, large campuses, military bases, etc. basically anything that would contain a number of individual networks which in turn contain a number of devices. Superclusters are collections of clusters, or put simply, countries.
Using carefully combined real world data, I've added every country in the world to the game (bar a few small island states with little or spurious data), and the largest ~4,500 cities in the world. The scale of a supercluster is determine using a combination of the countries total population and its GDP (which I currently use as a rough indicator of network density). Though I have also collected data on every country's internet penetration and total internet users, which I will find some use for I'm sure. The benefit of generating the world like this is that it allows me to defer the generation of actual networks until the player actually visits the cluster they are contained in. This gives me an effective order of magnitude increase in generation speeds.
While I only know the 'estimated' total world population of networks, devices, and users at generation-time, in practice, the player will not notice their absence before they are generated. Of course, to give the world some life and activity outside of the cluster the player begins in, I've selected a number of organizations (more on these later) in the world at generation-time and fully generated them and their employees (again, more on these later). This gives the world the illusion of being full of life and activity before the player visits more of its clusters on their travels.
Speaking of travels, a lot of work has gone into how this mechanic works. The player can only navigation around the world at a certain rate, which is plenty fast enough for navigating netween the networks in a cluster, but the distances between clusters is very large, and the distance between superclusters is vast. The 'connections' between clusters now act as a sort of highway, when the player approachs a connection, anywhere along its length, they are 'sucked' in and accelerating to great speed, pulling them to the distant connected cluster or supercluster. They can of course pull themselves out of the connection at any point, and I've got some plans for gameplay reasons for them to do so coming in the future. The current graphics for this are placeholder, but I have been playing around with some effects which I hope to incorporate soon:
Another benefit of travel via connections comes in world generation. The time spent travelling down a connection to a previously unvisited clusters gives me the time to generate that cluster before the player arrives, while the result still needs some graphics work, the effect will basically allow for seemless travel around a world in constant generation.
Dividing the game world up into clusters and superclusters will also allow me to scale the AI simulation around the world based on where the player is currently located. For instance, the player really doesn't need to know or care that Phil Robonson arrived 13 minutes late for work this morning and thus received a warning from his boss (more on this later), or that Sarah Peters learnt that her best friend is in serious debt (keep reading). But they certainly will care that the network administrator of the network they are in the middle of hacking just got paged after you triggered an alarm on the network's external firewall and is logging in to track you down. I still have to keep track of all the little details around the world, but most of them aren't that imporant from moment to moment and so can be batch simulated when they become relevant to the player. The player doesn't really care WHEN Robert Johnson from accounting got fired for fighting with one of his colleagues, they just need to know it happened if he becomes important to them later down the line.
As you might have realized, the simulation of the world has become quite a lot more detailed since I last wrote. HTP has taken an interesting turn in terms of gameplay, one I hope will further make it stand out in the small genre of hacking games. The level of detail in the simulation of the inhabitants of the world has been greatly increased. The anecdotes in the previous paragraph weren't just some hypothetical examples, they are all fully implemented and have real consequences to the player and all of the world's AI entities. As briefly mentioned earlier, I've added organizations to the game. Previously every network existed as an island of sorts, with no kind of controlling entity or real relationships to other entities (By 'entity' I mean anything tangible that interacts with other entities in the world. Organizations, networks, devices, people, etc.). Now when generated, networks are empty sites that are then allocated to an organization. Organizations vary in size, they can be small independent companies controlling a single network up to huge multinationals controlling hundreds of networks across the world.
When an organization sets up shop in a network they will procede to hire any employees they need to do what they do, this process involves looking at each position that needs to be filling in the network depending on what the organization does: administrators, programmers, bank managers, lawyers, managers, a CEO, etc. For each position, people will be randomly generated into the world, and in a kind of interview process, eventually someone with the appropriate personality traits will be selected (managers for instance, are required to be more demanding of their subordinates, and programmers are required to be of a certain level of intelligence). Once a suitable candidate has been found for the position, they are hired into the organization and are allocated to the network they will work in (not given a computer yet mind you, this is all done at cluster-level generation, not network-level, where devices are generated).
So now we have a bunch of employees working specific jobs at a specific place for a specific organization, we're half way there. These people don't just live to work, they must have lives outside of work as well. The next step is to find them a place to live. The generation engine next finds a home network somewhere nearby where they work and allocates them as a resident of said network. What about human relationships? Some people are single, some have partners, some are married, some are straight, some are gay, some are involved with people at work, some with people outside of work. All this is generated, so we end the cluster-level generation with thousands upon thousands of simulated people with their own personalities, lives and knowledge.
Knowledge. This is possibly the most important stage of generation. It's all well and good to have thousands of people going about their lives, but information is power, and information is the currency of HTP, and what information is there is no one knows anything? These most interesting part of developing this procedurally generated world for me has been simulating the accumulation and exchange of knowledge by the inhabiting entities. This process might be best described with a story. So I'm going generate a world right now with a random seed of 1632 (because it's 4:32pm currently), find a randomly selected person living in Vancouver, Canada (because that's where I live), let the world simulate for 1 month, and tell you their story.
- Our story begins with a 24 year old straight female named Jean Vaughn.
- She works at Central Telecommunication as an account manager.
- She lives at West Summit Plaza at 168 Hickory Crescent.
- She is in a relationship with Sean Brock, a 34 year old unemployed man.
- Jean's mobile number is (+1)681-6921.
- Jean's work computer's login is jean:jack.
- Jean is good at her job (with skill of 0.81), very ambitious (0.97), innoffensive (0.17), uncharismatic (0.16), loyal (0.74) and immoral (0.1), this will affect her actions.
- She is friends with her boss Megan Landry, a charismatic but somewhat confrontational 31 year old woman.
- Jean was recently given a verbal warning for being 18 minutes late for work.
- 58 other people live in the same apartment block as Jean.
- She lives on the 2nd floor, as well as 16 other people.
- Living next door to her is a 28 year old man named Raul Howe.
- He works at a Colocation run by North Telecommunication Co.
- North Telecommunication Co. is a small independent telecomms provider based out of Vancouver, Canada with 58 employees.
- While small the are very efficient and their last month's revenue was $3.3 million.
- Raul's work computer's password is 'ladybug'.
- It turns out Raul is working undercover for another organization, a secret he's so far kept to himself.
- Speaking of secrets, it also appears at some point in her past, Jean Vaughn hired a hitman.
I could go on, or let some more time pass and watch Jean and her friends' lives change as the world carries on, but I think this gives an idea of what's going on in the world at this stage. As a side note, the simulation of this world for a month took approximately 10 seconds.
What has made the development of this much detail possible is a tool I've been working on for the past 2 or so months. It's a standalone simulator of the game world that makes it much easier to see the vast amount of activity and information that the world simulation generates without all the overhead and obfuscation that the game engine itself creates. I've pulled all of the game logic out into a separate DLL that will let me run the game itself and the simulator on exactly the same code.
I sometimes wonder if spending so much time away from the game itself to create this simulator is worth it, then I look at how far the world simulation has come in the past 2 months, and how hard it would have been to make the same progress if limited by the game engine. I feel it has been completely worth it, and has certainly made it an order of magnitude easier to find and fix hard to spot bugs, as well as create the extremely high level of detail that will create a believable and interesting game world.
I hope this brings things back up to date, I'm sure I've not mentioned a ton of stuff I've added since April, but this is enough of a wall of text already and I don't want to overdo it.
As always... Hack the Planet!
Thursday 9 April 2015
Still Hacking
Well so much for keeping up with the blog. Every spare hour I've had over the past 2 months I've been spending on development so writing about it has taken been taking quite the back seat. I honestly don't know how indie developers find the time to keep dev blogs up to date. Well in news, I've had a chat with my boss about my day job, and I'm going to be cutting back from full-time to part-time, something almost unheard of in the games industry in my experience. This should give me not only at least another 20 hours a week to spend on HTP, but hopefully more time to keep this blog up to date and even, heaven horbid, have some free time!
HTP has come a long way since I last wrote, not only in terms of gameplay, but in visuals. I've replaced the previous placeholder UI with a 3D UI that whilst more or less functions the same way, looks much better. I always try to tell myself that graphics aren't that important and can just be added last, but it's hard to keep your morale up when all the other games you play look so much better than your own. I probably spend about a month completely redoing the UI engine and even though it has hasn't changed the gameplay much, it just *plays* better now. It also looks and feels more 'cyberspacey' than before.
(Please excuse the low quality/huge download size of the gifs, I'm new to this)
This isn't to say I haven't been working on gameplay at all. I've done a lot of work on the AI. Procedurally generated users now have lives, they go to work, they go out for lunch, they hang out in bars after work, and they go home to relax and sleep. They bring their mobile devices, connecting to networks with wifi as they move around, and have different routines depending where they are. If you, for instance, were to hack into a system administrator's workplace's sprinkler system while they were at work, and schedule a sprinkler test, they would be evacuated from the building and thus the network, leaving it more vulnerable to attack. You will even be able to get someone fired by planting illegal files on their machine and notifying the authorities. There is still much work to be done here, but the design is shaping up quite nicely to allow for some very interesting emergent gameplay.
I'm pretty happy with the visuals and UI at the moment, and I don't think I'll need to do much more work there for a while, leaving me free to get my teeth into the really juicy gameplay and AI behaviours. This is good, since it's something I'm much better at, and since very shortly I'll have much more time to spend on it, things should move very quickly. Exciting times
(I love how the 3D UI benefits from the crash post-effects in such an awesome way)
Thats all for now, once I move to part-time work over the next week or two, I'm going to try and plan my more plentiful time and make sure to allow for some hopefully weekly time to document my progress here.
Hack the Planet!
HTP has come a long way since I last wrote, not only in terms of gameplay, but in visuals. I've replaced the previous placeholder UI with a 3D UI that whilst more or less functions the same way, looks much better. I always try to tell myself that graphics aren't that important and can just be added last, but it's hard to keep your morale up when all the other games you play look so much better than your own. I probably spend about a month completely redoing the UI engine and even though it has hasn't changed the gameplay much, it just *plays* better now. It also looks and feels more 'cyberspacey' than before.
(Please excuse the low quality/huge download size of the gifs, I'm new to this)
This isn't to say I haven't been working on gameplay at all. I've done a lot of work on the AI. Procedurally generated users now have lives, they go to work, they go out for lunch, they hang out in bars after work, and they go home to relax and sleep. They bring their mobile devices, connecting to networks with wifi as they move around, and have different routines depending where they are. If you, for instance, were to hack into a system administrator's workplace's sprinkler system while they were at work, and schedule a sprinkler test, they would be evacuated from the building and thus the network, leaving it more vulnerable to attack. You will even be able to get someone fired by planting illegal files on their machine and notifying the authorities. There is still much work to be done here, but the design is shaping up quite nicely to allow for some very interesting emergent gameplay.
I'm pretty happy with the visuals and UI at the moment, and I don't think I'll need to do much more work there for a while, leaving me free to get my teeth into the really juicy gameplay and AI behaviours. This is good, since it's something I'm much better at, and since very shortly I'll have much more time to spend on it, things should move very quickly. Exciting times
(I love how the 3D UI benefits from the crash post-effects in such an awesome way)
Thats all for now, once I move to part-time work over the next week or two, I'm going to try and plan my more plentiful time and make sure to allow for some hopefully weekly time to document my progress here.
Hack the Planet!
Saturday 7 February 2015
Almost a Game
So a lot of time has past since my last update. But this time I'm not going to blame being busy at work, or Christmas, because I've in fact been working very hard on HTP. Every day for the past 98 days according to Github in fact. Look, I've got proof!
I've been carrying on implementing the tutorial, with a few interesting detours along the way. I've added so much in fact I don't know where to start. Maybe a quick run down on what the tutorial contains will help, so here it is so far:
Implemented:
- Intro
- Give the player their computer
- Port scan a neighbouring computer
- Identify the neighbouring computer
- Hack the neighbouring computer with the provided OS exploit
- Install the provided spam bot
- Go to a neighbouring home network
- Port scan and identify the router
- Reveal vulnerability database on the map
- Download a remote admin vulnerability for the router
- Start developing their own exploit from the vulnerability
- Speed up time until the development is complete
- Return to the neighbouring network and hack the router
- Scan the network to reveal computer and wifi access point
- Identify the switch and wifi access point
- Hack the switch
- Scan the network behind the switch
- Find the computer with the target user behind the switch
- Hack the computer using the OS exploit
- Reveal a software vendor on the map
- Purchase a key logger from the software vendor
- Install a key logger on the computer
- Identify the target phone
- Download a sensitive target file from the mobile using the provided OS remote copy file exploit
- Reveal a data exchange on the map
- Sell the sensitive file on the data exchange
Not yet implemented:
- Return to the keylogged computer and download the newly placed cryptocoin wallet
- Return to the home computer and open the wallet using the keylogged password
- Profit!
There will of course be much more to the gameplay then what is included here, but these will make up the basics, with more advanced gameplay being introduced throughout the storyline.
I've added externally accessible device content, or in other words, websites. These are device's contents that can be accessed without hacking into a network first, and are currently visible as cubes jutting out of the surface of networks.
There are 3 types of websites at the moment. Software vendors, where there player can purchase software. Data exchanges, where the player can buy and sell files (this includes all kinds of files at the moment, including illegally obtained files, but this will be replaced with blackmarket websites in the darknet eventually), and vulnerability databases, where the player can find publicly available vulnerabilities and develop them into usable exploits.
This brings me to the next feature, players can now create exploits from vulnerabilities. These exploits take some time to develop (time acceleration has also been added) and target specific versions of software. Different vulnerabilities have different effects on the target device, some will allow you to remotely copy a file, some will disable the device entirely, and some will grant you full access to the machine, or 'root' access as it's called.
I also completely rewrote the UI system to be totally data-driven, I did this over the Christmas holidays, it was quite boring, but needed to happen, and the sooner the better, as now adding now UI is as simple as whipping up a new JSON file along the lines of this:
{
"Elements":
[
{
"Name": "Divider",
"Layout": "=================================="
}
],
"Arrays":
[
{
"Name": "Cheats",
"Items": ["Complete All Missions", "Lotsa Money", "Complete All Development", "Toggle Instant Processes", "Skip Tutorial", "Unhack Target"],
"Layout": "{ITEM}",
"MenuNavigation": true,
"ActionsOnSubmit":
[
{
"Action": "RunCheat",
"Parameter": "{SELECTION}"
},
{
"Action": "GoBack"
}
]
}
],
"Layout": "Cheats\n{Divider}\n{Cheats}",
"Links":
[
{
"Keys": ["Escape", "Backspace"],
"Actions":
[
{
"Action": "GoBack"
}
]
}
]
}
I've also moved to Unity 5 pro, courtesy of a 90 day beta license gifted to me at a Unity 5 workshop that happened here in Vancouver in December, thanks guys!
Most of the first half of January was spent totally refactoring how software works. Before each type of program had its own source file which defined its behaviour. It worked but wasn't that flexible, now everything is, you guessed it, data-driven. The behaviours and development lifespan of a software product line are all defined in JSON files which look like the following:
{
"Name": "ScanX",
"Category": "Scanner",
"Description": "Brute force port scanner, not very stealthy but gets the job done.",
"Architecture": "x86",
"Tags": ["PortScanner"],
"StartingVersion": 1.0,
"VersionStep": 0.5,
"StartingLevel": 0,
"StartingCost": 50,
"LevelStep": 10,
"MaxReleases": 20,
"CostStep": 5,
"FormatString": "{NUMERIC:F1}",
"ExecutedActions":
[
{
"Action": "ScanPorts",
"MinSpeed": 5, "MaxSpeed": 2.5,
"MinNoise": 10, "MaxNoise": 10,
"MinRequiredLevel": 0, "MaxRequiredLevel": 5
}
]
}
You'll notice mention of 'level' a couple times in there. The game now has a global tech level which slowly increases over time, when the level increases enough, a software's developer will release a new version of it to the public, which can then be purchased (or otherwise 'aquired') and installed, reverse engineered, cracked, etc. I want to make the release and acquisition of things as dynamic as possible. There should be nothing to stop a hacker breaking into a software developer or vendor's network and stealing their latest, possibly even unreleased, software and selling it on the black market. Hackers don't sign NDAs right? Alternatively, if you could get your hands on an unreleased operating system and find a vulnerability in it, you could be sitting on a pretty juicy zero day exploit. But why stop there, maybe if you can create a stealthy enough virus, you could infect said unreleased OS and find yourself will a botnet of thousands created for you. It's that kind of lateral thinking and emergent gameplay I want to make possible and encourage. I want to provide the tools and let the players create their own fun.
My initial goal was to complete the tutorial and get HTP to the stage I can honestly call it a game by my birthday later this month. I'm actually a little ahead of schedule on the tutorial and it should be finished in the next couple days. But HTP doesn't quite feel like a game yet, it's still quite stagnent and static, so I think I'm going to keep myself on my toes and revise that goal to include added some AI to the game. I want to see freelance whitehat hackers aquiring software, finding vulnerabilities in it, and releases those vulnerabilities to the public in vulnerability databases. I want to see blackhat hackers hacking into networks and devices in search of cryptocoin wallets, installing spam and keyloggers, and generally causing mischief. I also want to see average users just going about their life, spending their day at work, and their evenings at home, playing games, shopping online, downloading porn, etc.
I'll leave you with a random image to add a bit of colour. I'll try to update this more frequently so I don't have to pour though my github commit logs to remember what I've actually done in the past 2 months!
Hack the Planet!
I've been carrying on implementing the tutorial, with a few interesting detours along the way. I've added so much in fact I don't know where to start. Maybe a quick run down on what the tutorial contains will help, so here it is so far:
Implemented:
- Intro
- Give the player their computer
- Port scan a neighbouring computer
- Identify the neighbouring computer
- Hack the neighbouring computer with the provided OS exploit
- Install the provided spam bot
- Go to a neighbouring home network
- Port scan and identify the router
- Reveal vulnerability database on the map
- Download a remote admin vulnerability for the router
- Start developing their own exploit from the vulnerability
- Speed up time until the development is complete
- Return to the neighbouring network and hack the router
- Scan the network to reveal computer and wifi access point
- Identify the switch and wifi access point
- Hack the switch
- Scan the network behind the switch
- Find the computer with the target user behind the switch
- Hack the computer using the OS exploit
- Reveal a software vendor on the map
- Purchase a key logger from the software vendor
- Install a key logger on the computer
- Identify the target phone
- Download a sensitive target file from the mobile using the provided OS remote copy file exploit
- Reveal a data exchange on the map
- Sell the sensitive file on the data exchange
Not yet implemented:
- Return to the keylogged computer and download the newly placed cryptocoin wallet
- Return to the home computer and open the wallet using the keylogged password
- Profit!
There will of course be much more to the gameplay then what is included here, but these will make up the basics, with more advanced gameplay being introduced throughout the storyline.
I've added externally accessible device content, or in other words, websites. These are device's contents that can be accessed without hacking into a network first, and are currently visible as cubes jutting out of the surface of networks.
There are 3 types of websites at the moment. Software vendors, where there player can purchase software. Data exchanges, where the player can buy and sell files (this includes all kinds of files at the moment, including illegally obtained files, but this will be replaced with blackmarket websites in the darknet eventually), and vulnerability databases, where the player can find publicly available vulnerabilities and develop them into usable exploits.
This brings me to the next feature, players can now create exploits from vulnerabilities. These exploits take some time to develop (time acceleration has also been added) and target specific versions of software. Different vulnerabilities have different effects on the target device, some will allow you to remotely copy a file, some will disable the device entirely, and some will grant you full access to the machine, or 'root' access as it's called.
I also completely rewrote the UI system to be totally data-driven, I did this over the Christmas holidays, it was quite boring, but needed to happen, and the sooner the better, as now adding now UI is as simple as whipping up a new JSON file along the lines of this:
{
"Elements":
[
{
"Name": "Divider",
"Layout": "=================================="
}
],
"Arrays":
[
{
"Name": "Cheats",
"Items": ["Complete All Missions", "Lotsa Money", "Complete All Development", "Toggle Instant Processes", "Skip Tutorial", "Unhack Target"],
"Layout": "{ITEM}",
"MenuNavigation": true,
"ActionsOnSubmit":
[
{
"Action": "RunCheat",
"Parameter": "{SELECTION}"
},
{
"Action": "GoBack"
}
]
}
],
"Layout": "Cheats\n{Divider}\n{Cheats}",
"Links":
[
{
"Keys": ["Escape", "Backspace"],
"Actions":
[
{
"Action": "GoBack"
}
]
}
]
}
I've also moved to Unity 5 pro, courtesy of a 90 day beta license gifted to me at a Unity 5 workshop that happened here in Vancouver in December, thanks guys!
Most of the first half of January was spent totally refactoring how software works. Before each type of program had its own source file which defined its behaviour. It worked but wasn't that flexible, now everything is, you guessed it, data-driven. The behaviours and development lifespan of a software product line are all defined in JSON files which look like the following:
{
"Name": "ScanX",
"Category": "Scanner",
"Description": "Brute force port scanner, not very stealthy but gets the job done.",
"Architecture": "x86",
"Tags": ["PortScanner"],
"StartingVersion": 1.0,
"VersionStep": 0.5,
"StartingLevel": 0,
"StartingCost": 50,
"LevelStep": 10,
"MaxReleases": 20,
"CostStep": 5,
"FormatString": "{NUMERIC:F1}",
"ExecutedActions":
[
{
"Action": "ScanPorts",
"MinSpeed": 5, "MaxSpeed": 2.5,
"MinNoise": 10, "MaxNoise": 10,
"MinRequiredLevel": 0, "MaxRequiredLevel": 5
}
]
}
You'll notice mention of 'level' a couple times in there. The game now has a global tech level which slowly increases over time, when the level increases enough, a software's developer will release a new version of it to the public, which can then be purchased (or otherwise 'aquired') and installed, reverse engineered, cracked, etc. I want to make the release and acquisition of things as dynamic as possible. There should be nothing to stop a hacker breaking into a software developer or vendor's network and stealing their latest, possibly even unreleased, software and selling it on the black market. Hackers don't sign NDAs right? Alternatively, if you could get your hands on an unreleased operating system and find a vulnerability in it, you could be sitting on a pretty juicy zero day exploit. But why stop there, maybe if you can create a stealthy enough virus, you could infect said unreleased OS and find yourself will a botnet of thousands created for you. It's that kind of lateral thinking and emergent gameplay I want to make possible and encourage. I want to provide the tools and let the players create their own fun.
My initial goal was to complete the tutorial and get HTP to the stage I can honestly call it a game by my birthday later this month. I'm actually a little ahead of schedule on the tutorial and it should be finished in the next couple days. But HTP doesn't quite feel like a game yet, it's still quite stagnent and static, so I think I'm going to keep myself on my toes and revise that goal to include added some AI to the game. I want to see freelance whitehat hackers aquiring software, finding vulnerabilities in it, and releases those vulnerabilities to the public in vulnerability databases. I want to see blackhat hackers hacking into networks and devices in search of cryptocoin wallets, installing spam and keyloggers, and generally causing mischief. I also want to see average users just going about their life, spending their day at work, and their evenings at home, playing games, shopping online, downloading porn, etc.
I'll leave you with a random image to add a bit of colour. I'll try to update this more frequently so I don't have to pour though my github commit logs to remember what I've actually done in the past 2 months!
Hack the Planet!
Tuesday 9 December 2014
Tutorial-Driven Development
I've come up with a new approach to development. Previously I would just choose the next feature I though HTP needed to slowly drive it towards gameplay completion, and tackle that. It was working, but it was slow and haphazard, also it resulted in a lot of the little details that make a game complete getting missed out.
My new approach is something I'm called "tutorial-driven development", I suppose it's a bit like test-driven development for games in a way. I started by planning out the entire tutorial sequence a new player will be presented with on the start of a new game. A sequence that when completed will give a new player a basic handle on the gameplay. It was a little intimidating to realize how much work was still left to do before I could call HTP a 'game' of any description. But it also gave me an excellent to-do list of large and small features I needed for that to happen. I've just been slowly chewing my through those features.
The first step was to create a mission chain system, since the tutorial would be made up as a series of missions, which might share common source users and target devices. For those interested, a mission chain template looks a bit like this:
{
"Contacts":
[
{
"Name": "A Friend",
"Type": "Hacker",
"IsMale": "false"
}
],
"Targets":
[
{
"Name": "NeighbouringComputer",
"DeviceTypes": ["Computer"],
"Vicinity": "HomeNetwork"
},
{
"Name": "NeighbouringNetworkRouter",
"DeviceTypes": ["Router"],
"Vicinity": "NeighbouringNetwork",
"Origin": "HomeNetwork"
}
],
"Chain":
[
{
"Description": "Welcome to the grid {ACCEPTINGUSERNAME},\n\nYou don't know me, but you can trust me. I was once like you, back when I was just getting started.\n\nI guess I feel for you.. I know how hard it can be, at the beginning. I'm going to help you, cause a long time ago, someone helped me, so it feels like the right thing to do. Besides, I could use a little good karma right about now.\n\nReply to this message and we can get started.\n\n - {SOURCEUSERNAME}",
"ShortDescription": "Welcome",
"ResponseMessage": "Let's get started.\n\n - {ACCEPTINGUSERNAME}",
"Goal":
{
"Type": "Respond",
},
"Source": "A Friend",
"DelayUntilNextMission": 2
},
...
I won't get into too much detail explaining this, as I'm trying to keep this update more gameplay-oriented, but I'm sure the coders out there can see where I'm going with this. Unfortunately these mission chain templates, unlike many of the other templates, are getting quite clunky to edit by hand, so at some point I think I'm going to have to pull my thumb out and write an editor for all these templates. I'm mainly putting it off cause it's boring as hell.
I've completed the first 9 tutorial missions so far, and in the process have been forced to code lots of little tiny features and polish existing ones, it's been a great exercise.
Another thing I've been working on has been revamping the connections between nodes. Until recently they were just simple 1 pixel wide lines that flashed with network activity, I always wanted them to be more then that, so to that end I've replaced them with true cylindrical meshes. Now now have real dimensions, and will be able to contain things like neat traffic effects, or even the player, acting as kinds of highways.
Obviously a lot is still to be done with their visuals, but I've proven the performance hit of having thousands of true 3D connections instead of simple lines is negligible.
Well that's all for now, just a quick update. The next thing I will be working on is letting web servers has a role defined, which governs what kind of web content they will serve. Starting with a vulnerability database needed for the next tutorial mission.
Hack the Planet!
My new approach is something I'm called "tutorial-driven development", I suppose it's a bit like test-driven development for games in a way. I started by planning out the entire tutorial sequence a new player will be presented with on the start of a new game. A sequence that when completed will give a new player a basic handle on the gameplay. It was a little intimidating to realize how much work was still left to do before I could call HTP a 'game' of any description. But it also gave me an excellent to-do list of large and small features I needed for that to happen. I've just been slowly chewing my through those features.
The first step was to create a mission chain system, since the tutorial would be made up as a series of missions, which might share common source users and target devices. For those interested, a mission chain template looks a bit like this:
{
"Contacts":
[
{
"Name": "A Friend",
"Type": "Hacker",
"IsMale": "false"
}
],
"Targets":
[
{
"Name": "NeighbouringComputer",
"DeviceTypes": ["Computer"],
"Vicinity": "HomeNetwork"
},
{
"Name": "NeighbouringNetworkRouter",
"DeviceTypes": ["Router"],
"Vicinity": "NeighbouringNetwork",
"Origin": "HomeNetwork"
}
],
"Chain":
[
{
"Description": "Welcome to the grid {ACCEPTINGUSERNAME},\n\nYou don't know me, but you can trust me. I was once like you, back when I was just getting started.\n\nI guess I feel for you.. I know how hard it can be, at the beginning. I'm going to help you, cause a long time ago, someone helped me, so it feels like the right thing to do. Besides, I could use a little good karma right about now.\n\nReply to this message and we can get started.\n\n - {SOURCEUSERNAME}",
"ShortDescription": "Welcome",
"ResponseMessage": "Let's get started.\n\n - {ACCEPTINGUSERNAME}",
"Goal":
{
"Type": "Respond",
},
"Source": "A Friend",
"DelayUntilNextMission": 2
},
...
I won't get into too much detail explaining this, as I'm trying to keep this update more gameplay-oriented, but I'm sure the coders out there can see where I'm going with this. Unfortunately these mission chain templates, unlike many of the other templates, are getting quite clunky to edit by hand, so at some point I think I'm going to have to pull my thumb out and write an editor for all these templates. I'm mainly putting it off cause it's boring as hell.
I've completed the first 9 tutorial missions so far, and in the process have been forced to code lots of little tiny features and polish existing ones, it's been a great exercise.
Another thing I've been working on has been revamping the connections between nodes. Until recently they were just simple 1 pixel wide lines that flashed with network activity, I always wanted them to be more then that, so to that end I've replaced them with true cylindrical meshes. Now now have real dimensions, and will be able to contain things like neat traffic effects, or even the player, acting as kinds of highways.
Obviously a lot is still to be done with their visuals, but I've proven the performance hit of having thousands of true 3D connections instead of simple lines is negligible.
Well that's all for now, just a quick update. The next thing I will be working on is letting web servers has a role defined, which governs what kind of web content they will serve. Starting with a vulnerability database needed for the next tutorial mission.
Hack the Planet!
Saturday 6 December 2014
Saturday 29 November 2014
Thin Vertical Slice
It's not even been a month yet, I feel like I'm jumping the gun with another update. But I've got so much done over the past few weeks I feel like if I left it any longer it would result in my brain exploding into an endless wall of text no one would care to read.
I'm going to present this in more of a bullet point form, as there are a bunch of small and large things I want to document here, so in more of less chronological order since since my last update:
Next thing I want to do with users is give them presence, so they can either be at work or at home, and depending on their job and function, this will effect gameplay. If a extremely skilled security officer in a corporate network is giving you grief, simply find out where he lives, hack into his home network, and set off his security alarm, causing him to go home and getting him off your back.
I've also replace the old camera controls with a new physics based camera. I think it feels much more satisfying to get around with these new controls. The camera accelerates, drifts, can spin and roll and interacts with the world more realistically. I also added a neat impact effect using the new post-effects (more on that later):
Once inside a network, they are kind of self-contained little levels, which you fly around and hack your way through. Still a lot of work to do here, I want to have the outside world sill visible through the boundaries of the network, though slightly blurred. And I would love to differ the style and appearance of varying types of networks. Cold, monolithic regimented looks for military and government networks, and warm, cozy, loose looks for home networks.
Well that's it for now, I think I've got enough off my chest to carry on with the next push. I think the next thing on my plate is giving money a purpose and adding 'shops' to the game. Don't know how they're going to work yet, but I'll figure something out.
Now if you'll excuse me, there's 23 punk bands playing tonight for $15 and I need to get my drink on...
Hack the Planet!
I'm going to present this in more of a bullet point form, as there are a bunch of small and large things I want to document here, so in more of less chronological order since since my last update:
Users:
Users are tangible things now, they have names, skills, homes and workplaces. They come in a variety of types, some separated by income brackets, some by affiliation. Different kinds of users will generate different amounts of network traffic, and their files and network traffic have different values based on what kind of user they are. A hacker will generate gobs of network traffic of a fairly high value, but has an extremely high security skill rating, which will make it much harder to steal from them. A low income average user has a lower security rating and lower value files and traffic.Next thing I want to do with users is give them presence, so they can either be at work or at home, and depending on their job and function, this will effect gameplay. If a extremely skilled security officer in a corporate network is giving you grief, simply find out where he lives, hack into his home network, and set off his security alarm, causing him to go home and getting him off your back.
Movement:
I always wanted to tie movement into the connections between networks. I still want to maintain the feeling of flying freely through cyberspace, so I've compromised and calculated the player's speed based on their proximity to network connections and how much bandwidth those connections have. The result will be that a player must more or less follow the connections between networks, and faster connections will act as highways between distant networks.I've also replace the old camera controls with a new physics based camera. I think it feels much more satisfying to get around with these new controls. The camera accelerates, drifts, can spin and roll and interacts with the world more realistically. I also added a neat impact effect using the new post-effects (more on that later):
Traffic Visualization:
Network traffic is visible now on the connections between networks. Depending on that kind of devices are connected, and who is using those devices, different amounts of traffic is generated. A printer talking to a home network's switch will create very little traffic, whilst a web server connected to a firewall will create a significant amount of traffic. These effects can be seen in the video below, and are made all the more obvious with the new bloom effect (again, more on that later):Network Entrances:
Networks are now contained spaces, with entrances consisting of devices like routers. You can 'pass through' a network by flying around it, but if you want to access the internal devices in the network, you first have to gain access to one of these entrances.Once inside a network, they are kind of self-contained little levels, which you fly around and hack your way through. Still a lot of work to do here, I want to have the outside world sill visible through the boundaries of the network, though slightly blurred. And I would love to differ the style and appearance of varying types of networks. Cold, monolithic regimented looks for military and government networks, and warm, cozy, loose looks for home networks.
Post-Effects:
The Unity asset store takes most of the credit for these improvements. I found an excellent old TV tube shader to create the scanline, noise and chromatic aberration effects (check it out here: https://www.assetstore.unity3d.com/en/#!/content/24046). I also added some bloom, glow, vignette and FSAA effects to some cheap graphics wins. The improvement in appearance should tide me over for a while and make me less embarrassed to show off what I have so far.Messaging System:
This was going to start out as a tutorial system for new players, but I think will become much more integral to gameplay in the future. It's basically an email inbox. When writing the initial introduction message, it became apparent that a storyline could very well fit into this game, and as much as I don't really care for storylines in games, I do kind of look forward to writing one, no matter how flimsy, for HTP.Missions:
After creating a basic messaging system, I got a bit carried away and ended up writing a random mission generation system. I didn't really plan on having missions in HTP, but after having added them, I'm not sure how it would have ever worked without them. Here's a short video running through a simple 'steal the file' mission. The mission is only completable with debug tools, but it gets the idea across:Name Generation:
Finally, after having added missions, and not wanting to simply waypoint the mission's goal out blatantly, I added random network name generation to the world. This will help me guide the player to a mission's target without directly revealing it, "Please steal file X from Mrs. Y, she lives near Z.". This was pretty fun to implement, I do love seeing procedural generation in the wild :) Though I have to say, most of the business names come off as a little 'new media' at the moment :SWell that's it for now, I think I've got enough off my chest to carry on with the next push. I think the next thing on my plate is giving money a purpose and adding 'shops' to the game. Don't know how they're going to work yet, but I'll figure something out.
Now if you'll excuse me, there's 23 punk bands playing tonight for $15 and I need to get my drink on...
Hack the Planet!
Subscribe to:
Posts (Atom)