Mixing synchronous and asynchronous requests for serious speed

You know what I’m talking about. Those infuriating websites that present animated grey boxes while they fetch their content asynchronously. For seconds. No one has seconds. Give me the content now!

Draft Rugby is a fantasy Rugby app. It’s in early development. Our main feature is the Player Stats page. This page is effectively a glorified table: It allows rapid search and sorting of the ~800 players in the Super Rugby season.

Before today, it loaded pretty quickly. Here’s the process:

  1. A browser makes a GET request to /player-stats
  2. Draft Rugby replies with a bunch of HTML, CSS, and JS
  3. The browser runs the JS, which includes an immediate GET request to /api/fantasy/player/list vis the Draft Sport JS library
  4. Draft Rugby replies with a bunch of JSON
  5. The browser eats the JSON and fills the player table

Step 3 is an asynchronous javascript request leading to document manipulation, commonly known as “AJAX”. That’s nice, because the user can now sort and search the table. Each time they do, more asynchronous requests are made to get them the data they want, and refill the table.

Except it’s not always nice, because of the speed of light. In an ideal case, with a client device say, 30 kilometres from the datacenter, there might be 50 milliseconds between the start of step 1 and the start of step 3. In Draft Sport’s case it was taking a whopping ~270ms to finish the whole sequence and begin animating the table.

Alt Text

No one has time for 270ms! A user will notice this delay, without question. And it get worse: Your user’s visual processing system needs to parse your fancy loading animation while the async request is happening. Then it needs to dump that information and re-parse the actual page content.

Don’t do this! It sucks! Let’s shift the initial table load back onto the server. Now the sequence looks like this:

  1. A browser makes a GET request to /player-stats
  2. Draft Rugby replies with a bunch of HTML, CSS, and JS, including the content of the player stats table retrieved via Draft Sport Py.
  3. The browser paints everything

From 5 steps to 3. Now:

  • No double round-trip to the datacenter to fetch the initial data
  • No loading animations for the user to parse

What’s the tradeoff? It depends on the disposition of your API. Draft Sport API is not the fastest thing in the world, yet – It takes about 50ms to retrieve the player table. That request now blocks the time-to-first-byte, slowing the page delivery down by 50ms.

The synchronous result is still way better. The time until content is fully presented drops from about ~450ms to ~200ms. As Draft Sport API matures and gets faster, that time will drop further, whereas the speed of light isn’t going anywhere. And in the real world, your user is not going to be 20ms from your datacenter. The further away they are, faster the synchronous request becomes. Your framework can’t outrun the speed of light!

Conclusion? Don’t be afraid to hold up returning your first byte with a server-side API request. If you know what data the client wants, your overall time to displayed content will probably be significantly lower than if you return it asynchronously via an AJAX request.


Cognition-oriented Programming

Programming is addictive. As programmers we have immense power. We can create machines of practically unlimited scale and scope. Our creations may yield immense joy and utility. In code anything seems possible.

We programmers are still human. We are delicate, non-deterministic machines with squishy bodies and even squishier minds. Our capacity to build software is a function of our capacity to direct the limited resources of our bodies and minds.

Programming paradigms provide loose rulesets to help us direct those resources. We adopt functional-oriented, object-oriented, data-oriented, or some other programming orientation. These paradigms are well established and serve wonderfully.

They are top-level distinctions that sit on top of a huge stack of programming techniques, patterns, fads, and principles. Much joy comes from exploring new ways to write code, and finding out what allows each of us to most effectively reach our goals.

Science gets us some of the way

What should drive that exploration? How can we know whether we should adopt a data-oriented approach to our latest project? The decision is often made out to be one of computer science. What paradigm best fits the requirements of the system?

A game programmer might generally require their program to respond to user input in 16 milliseconds or less. They might be well served in trading some readability for intensely optimised hot-loops that keep as much data in L1 cache as possible. Perhaps data-oriented C++?

A web programmer might have bit more room to move, client-side progress animations and common acceptance of network latency yielding an acceptable response time of 20 – 150 milliseconds. They might be well served optimising for readability. Perhaps object-oriented Ruby?

Yet the choice is rarely so clear cut. The latest fads, frameworks, and celebrity-programmer proclamations might influence us in different directions.

Think first, paradigm later

Our guide should be the disposition of our own cognition. Each of us has a different mind. Your brain has developed differently to mine. Different experiences, training, and genetic-predispositions conspire to influence the manner in which we reason about code.

This is quite abstract. Let us examine a practical example. Consider a single class definition. How long is too long? 200 lines? 1,000 lines? What if we mix multiple classes into a single file?

You might be comfortable working with files thousands of lines long. Having witnessed such capability, I initially worked in this manner as well. Over many years I realised that I was not capable of maintaining a mental map of so much data. Now, I aim for a maximum of 150 lines per file, with strictly one class definition per file.

If a definition becomes longer than 150 lines, I actively ponder breaking it into smaller composable parts. As I move through my programs, I operate on a maximum of two files at a time, working on the flow of data from to another. In this way, I am always operating on a maximum of approximately 300 lines of code.

To you, this might sound absurd. You might have huge files and be perfectly comfortable surfing through them. This is exactly the point: We are all different, and we must seek to write code in a manner that supports our own cognition. Call it “cognition-oriented programming”.

Cognition for Teams

In a team, this might appear to present problems. What if your CTO likes massive files and you don’t? I suggest that in a well-gel’d team, leadership is setting objectives and standards, not methods.

That is, you are ideally told what you need to do and the bounds within which you need to do it, not how to do it. In such an environment, you might adopt a team-wide code-style guide. You might have rules such as “no global variables” and “no mutable state”.

Then, within these bounds, you might focus on providing maintainable, reasonable, and clear public interfaces for your code. Behind those interfaces, and while always maintaining compliance with the set bounds, you may lean towards optimising for your own cognition.

Your Optimised Mental Model

The concept of being “in flow” is well studied. Cognition-oriented programming might be viewed as a subset of the requirements for being “in flow”. At the highest level, it may be viewed as the search for your own hyper-optimised mental model.

Your brain has some strict finite limits. Your short-term working memory is likely limited to storing around seven “objects” at once. Your long-term memory might store pointers to every corner of your 200,000 line program, but at any instant, you can only reason with reference to some small subset of that code.

How you structure that limited reference is crucial. Think of your working memory as your own L1 cache. In paradise, your program is crunching cache data at warp speed, never needing to reach out to comparatively glacial RAM. In maximum flow, your mind is crunching on concepts stored in your working memory set. You reason about them at immense speed and with great precision.

In reality, our real programs need to go to RAM. Worse, they might need to access persistent storage. Heaven-forbid they need to take the long voyage out through the network stack. This is true for our mental models as well. In our metaphor, RAM might be some other piece of our codebase. Storage might be referencing some documentation. Network I/O might be talking to another human being.

All of those operations are crucial and important. Optimising them will yield more time in nirvana, with your optimised mental model facilitating a flow of performant, reliable, and reasonable code. The feeling, in my experience, is close to that of playing a musical instrument. The dance of fingers across the keys is unending, rhythmic, and perhaps even spiritual.

Seek Nirvana

Finding your cognition-oriented model takes time. Try new patterns. Read widely. Observe what works for some very successful programmers. Not because you should imitate them, but because their models might provide inspiration for some part of your own.

Engage in trial and error. Construct, destruct, and modify systems. All the while, observe yourself. Do you feel transcendent? Or do you feel as though you are straining to keep a grip on the program? Do you instantly see the path to a new feature, fixed bug, or performance improvement? Or do you strain to comprehend how to proceed?

These questions will enable you to find the patterns that map to your own cognition. Do not force your patterns on others. Do not presume others’ patterns will work for you. Code is the tool that enables you, as a squishy mammal, to dent the universe. Seek to make it your own.

Une Petite Consolation

The Notre Dame represents a heart of Paris, and a beacon of civilisation. Cynics would call it a tourist trap. Even they could not ignore its sublime aesthetic and rich representation of human history.

In our wanderings of Paris, we might pass it without thinking. After visiting the cathedral once, it fades into the background, contributing to the delectable palette of the French capital’s core.

The loss of her great and aged timber is profoundly upsetting. Yet there is cause for confidence, for Parisians and all those who celebrate great artefacts everywhere.

In Japan, one regularly stumbles across buildings of immense historical value. Some are as old or older than the Notre Dame. Exploring them delivers a sense of profound wonder, the same sense of looking through time as one experiences in Paris.

Physically rebuilt, the wonder remains

Yet look closely, and deeper truth is revealed. Many Japanese buildings, even the most celebrated, such as Osaka Castle, bear plaques that read along the lines of:

“Constructed 1237, destroyed by fire 1642, rebuilt 1899”

It is the nature of life for things to be lost. No matter how careful a civilisation, it will always lose relics over time. What matters is not the physical form of the relic, but what it represents, how we handle loss, and whether we rebuild stronger than before.

Electric thoughts for my conservative loved ones

Australia’s political debate was recently given a high-voltage shock. The Morrison Liberal-National government was preparing to call an election. Bill Shorten was diligently keeping his mouth shut, knowing that Labor was on autopilot to a likely victory.

Then, in his budget reply speech, Bill mentioned electric cars. Of all the topics he raised, that one seems to spark the most discourse.

On that day, as I read the speech, I was sitting in an electric car, or “EV”, doing 120km/h on a Florida freeway. Like Bill, the car was quite literally on autopilot, driving itself entirely. My partner Jessica sat in the drivers seat, monitoring the car’s performance as it watched for humans doing crazy human things.

This particular freeway connects Trump Country and… the rest. Florida is a purple state, with gun-toting, gas-guzzling gated communities a short drive away from  urbane, medium density developments whose residents commute by train.

Out here, I have experienced first-hand the pain of families and friends being torn apart by the hysterical partisanship Trump cultivates. Healing these rifts is tough. Dinner conversations are delicate affairs, where third rails (immigration, healthcare, trade, and more) must be carefully avoided.

The EV debate in Australia has surprised me, because the tone feels like that which I’ve found in Trumpland. Rather than engaging with the science and economics of EVs, the debate appears to be morphing into a cultural one. Internal combustion engines (ICE) represent some sort of honourable status quo. EVs represent a fad, an “other”, at worst a conspiracy designed to screw the unwary out of their excellent ICEs.

At left, the Model 3 giving a graphical representation of its situational awareness. At right, the simultaneous view out the windshield

As the keeper of a V8 BMW, which I adore, and as someone who has lived with EVs on the East & West coasts of the United States, I can assure you that the conspiracy is operating in reverse.

EVs are, pound for pound, faster, safer, less complicated, more comfortable, cheaper to run, capable of carrying more cargo, and handle more nimbly than ICE cars. Anyone that tells you otherwise has not spent any significant time in a Tesla Model 3. Even the best German ICE engineering cannot overcome the fundamental design advantages of removing a huge, heavy lump of finely-tuned moving parts from the vehicle.

Regardless of whether they are good for the environment, EVs are simply better cars. Yet even when fed with electricity generated by the dirtiest coal power stations, they are vastly less carbon-intensive per unit of distance travelled.

Their only drawback, their sole weakness, is the price of their lithium ion batteries. Yet the price of these batteries is steadily falling, and the point at which they become cheaper than relatively immensely complicated ICE’s is inevitable.

Charging infrastructure is ubiquitous – Even a standard 110v plug in the US can comfortably deliver 7km/h of range at 10 amps, more than enough to cover a light ICE duty cycle. Installation of relatively inexpensive three phase wall chargers can deliver ~35km/h at home. The Tesla Supercharger network delivers an astonishing 800km/h and up, and they are everywhere, even in Australia. No matter where you charge, the price per km travel of electricity is far below that of petroleum.

Ubiquitous superchargers can deliver 800km/h in range

I am fortunate to be a digital nomad, criss-crossing the world and observing a new city almost every week. Compared to what I see on my travels, the dialogue in Australia around energy is like something out of the Stone Age.  Out here, in the rest of the world, the pace of change is astonishing. There is not a single petrol powered motorcycle left on Beijing’s streets. EVs are now the most popular premium vehicle in the United States and European Union. Car parks are full of chargers, while freeways are peppered with EVs driving themselves.

Meanwhile in Australia, smug News Corp pundits are making idiotic quips about EVs shortfalls from… the 90s.

If Bill Shorten wants to shave a few percentage points off my disposable income while he articulates a semi-coherent vision for jolting Australia out of its coal-induced stupor then yes, I’m ok with that. The rest of the world is moving on, at EV speed. Meanwhile, the Liberal-National coalition is running Facebook scare-campaigns promoting the primacy of the ICE.

We are fortunate in Australia to have two broadly competent major parties, who both have a history of “not screwing it up too badly” over the past 40 years. When one of them is running such an appalling backward looking campaign, it is time to give the other one a chance.

Time for Australia to plug in

The Joy of Testing

Unit testing is great because it helps us make better software. A good test suite will catch bugs, maintain stability, measure performance, and more.

Yet to be honest, those aren’t the reasons I like testing so much. I like writing tests because it just feels great to run them. Sometimes I catch myself watching the Amatino API test suite running, not because I need to test changes, but because I simply want to watch it run.

At upper left, we have the HTTP returns for the public-facing API. At lower left, the internal API. At right, the list of successful tests marching downwards.

It is the closest I can be to watching the machine think. All the thousands of lines of code, the weeks of agonising over tough problems, the frustrations of insidious bugs, the joys of successful output. All distilled into a real-time ballet of virtual machines, database queries, http requests, and code execution that I can see with my own eyes.

Write tests. Not because the bloggerverse told you to, but because it just feels good.

How to install OpenSSL & stunnel on MacOS

When travelling behind the Great Firewall of China,  I wanted a copy of OpenSSL and stunnel on my machine. Googling “install stunnel macos” gives a bunch of answers that involve the word “brew”.

OpenSSL and stunnel are open-source C programs, which means we can compile them from source. Doing so is not difficult, but it is a bit fiddly, and I think that fiddliness can dissuade people and cause them to unnecessarily reach for a bloated package manager.

All the information in this blog post is “as of February 2019”, and applies to stunnel 5.50, OpenSSL 1.1.1, and macOS 10.14.

stunnel depends on OpenSSL, so we will compile and install OpenSSL first.

Compiling & Installing OpenSSL on MacOS

stunnel is going to look for ssl.h, and to make it available we need to compile with the shared flag.  We probably don’t want to spray OpenSSL all over the system, so we will use --prefix to specify an install location other than the default /usr/bin.

$ ./Configure darwin64-x86_64-cc shared --prefix=/Users/hugh/somedir --openssldir=/Users/hugh/somedir no-ssl3

Now you can make and make install. Many lines of output later, you will

Compiling & Installing stunnel on MacOS

OpenSSL at the ready, we can now move on to stunnel.   We will tell stunnel about our newly minted copy of OpenSSL using the ./configure command:

$ ./configure --with-ssl=/Users/hugh/somedir --prefix=/Users/hugh/somedir

If the shared flag was not specified when compiling OpenSSL, then this is where you will hit the much-googled:

checking for TLS directory... not found
configure: error: 
Could not find your TLS library installation dir
Use --with-ssl option to fix this problem

We did used shared, so we are good to go! Hit that make button, followed by make install, and you are the proud owner of a copy of stunnel compiled with OpenSSL!

Immutable objects in Python

Python is a beautifully flexible beast. Class properties are variable, which means we can change them from anything to anything at any time.

class Flexible:
    piece = "hello world"

instance = Flexible()
print(instance.piece)  # prints “hello world”
instance.piece = 42
print(instance.piece)  # prints “42”

Sometimes, we might want to trade some flexibility for safety. Humans are fallible, forgetful, and fickle beasts. Programmers are also humans. We make mistakes more often than we would like to admit.

Fortunately, Python gives us the tools protect ourselves against ourselves. Where we want to, we can trade flexibility for safety. You might wish to protect yourself by creating immutable objects : Instances of a class that can’t be modified once they are created.

In this article, we will seek immutability of properties. That is, we will stop ourselves from being able to change the.piece property of a Flexibleclass.

By making our class properties immutable, we eliminate the need to reason about object state. This reduces our cognitive load, and thus the potential for error.

Note that the immutability in this context is different to immutability  from the perspective of memory, an equally valuable but different angle on the broad topic of immutability in general.

Our objective is to achieve immutability from the perspective of the programmer – To explicitly catch cases were we accidentally attempt to mutate a property that we should not. From the perspective of the machine, the property is still perfectly mutable. We aren’t trying to change the way the property behaves in memory, we are trying to protect ourselves from our own stupidity.

To create an immutable property, we will utilise the inbuilt Python property class. property allows us to define get and set behaviour for a property.

class Flexible:
   piece = property(lambda s: "hello world"w)

instance = Flexible()
print(instance.piece)  # prints “hello world”
Instance.piece = mutated  # throws AttributeError

The property class takes four parameters. The two we will focus on here are fget and fset. In the above example, lambda s: “hello world” was our fget, allowing us to print(instance.piece). The absence of fsetcaused the AttributeError when we attempted to set the value of instance.piece to ’mutated’.

An AttributeError might be a solid enough reminder to yourself that you’ve accidentally done something dumb. However, you might be working on a project with multiple programmers. Perhaps an AttributeError is not a clear enough warning to others that a property should not change.

For example, a colleague might interpret that AttributeError as a sign that you simply forgot to implement fset. They might merrily edit your class, adding fset, unknowingly opening a Pandora’s Box of state-related bugs.

To give our colleagues as much information as possible, let’s make the immutability explicit. We can do so by subclassing property.

class Immutable(property):
   _MESSAGE = "Object state must not be mutated"

   def __init__(self, get) -> None:

   def self._set_error(self, _1, _2) -> None:
       raise RuntimeError(self._MESSAGE)

Now, when we attempt to change a property, we get a clear and unambiguous error.

class Flexible:
   piece = Immutable(lambda s: "Can't touch this")

instance = Flexible()
instance.piece = "try me"  # Raises RuntimeError with clear description

Of course, a lambda serving a constant is not going to satisfy many requirements. You can supply the fget parameter something more useful. For example, suppose a class maintains some internal state, readable by the whole program. It is crucial to the safe operation of the program that nothing outside the class modifies that state.

class Flexible:
   _internal_state = 42
   some_state = Immutable(lambda s: s._internal_state)

In this case, the rest of the program can safely access the value of _internal_state via the some_state property. We provide a strong hint to our colleagues that _internal_state is off limits by using the leading underscore: A convention for hinting that a variable be treated as “private”. The value returned by some_state can be changed internally by the class, but it is very hard for a programmer to accidentally modify the state externally.

Other languages might achieve this behaviour in other ways, especially through the use of the private keyword. For example, in Swift:

class Flexible {
   public private(set) var some_state = 42

Unlike Swift and others, Python will not explicitly stop someone from modifying the Flexible state. For example, a colleague could easily execute

instance._internal_state = "where is your god now?"

That flexibility is a great strength of Python. The point is not to stop anyone doing anything. The point is to provide helpful hints, checks, and clues to stop ourselves from making silly mistakes.

Result types in Swift. Are they useless?

Fresh-faced, amateur, and impressionable: Swift is not my main jam. When setting out to write Amatino Swift, I was hungry for best-practice and good patterns. Amatino Swift involves lots of asynchronous HTTP requests to the Amatino API.

Asynchronous programming requires bifurcation at the point of response receipt. That is, an asynchronous operation may yield a success state or a failure state. Result types are a pattern widely espoused as a pattern for handling such bifurcation.

I’d like to open a debate over whether result types should be used in Swift. After some thought, it appears to me that they are useless. I propose that we would be better off encouraging newbies to utilise existing Swift features, rather than learning about and building result type patterns.

For the purposes of this discussion, let’s assume that our code includes two functions, each of which handle a bifurcated state:

func handleSuccess(data: Data) {
  // Do stuff with data

func handleFailure(error: Error) {
  // Do stuff with error

Inside these functions, we might implement code which is independent of our bifurcation pattern. For example, we could `case-switch` on Error type in order to present a meaningful message to a user.

Now to the bifurcation itself. A naive and simple pattern might be:

// This is bad code. Do not copy it!
func asynchronousCallback(error: Error?, data: Data?) -> Void {
  if (error != nil) {

There are myriad problems with this approach. We have no type safety over `data`. We do not control for cases where programmer error yields a state where both `data` and `error` are `nil`. It’s ugly. More.

Result types propose to improve upon this pattern by defining a type such as:

enum Result<Value> {
  case success(Value)
  case failure(Error)

Which may be used like so:

func asynchronousCallback(result: Result<Data>) {
  switch result {
  case .success(let data):
  case .failure(let error):

This pattern introduces type safety to both `error` and `data`. I suggest that it does so at too great a cost when compared to using inbuilt Swift features. Every asynchronous bifurcation now requires a `switch-case` statement, and the use of a result type.

Compare the result type pattern with one that uses the Swift `guard` statement:

func asynchronousCallback(error: Error?, data: Data?) {
  guard let data = data else { handleError(error ?? TrainWreck()) }; return

In this case, we have type safety over `error` and `data`. We have handled a case in which a programmer failed to provide an `Error` using the nil-coalescing operator `??`. We have done it all in two lines of less than 80 char. A suitable error type might be defined elsewhere as:

struct TrainWreck: Error { let description = "No error provided" }

Bifurcation via a `guard` statement appears to me to have several advantages over result types:

  • Brevity. Functions handling asynchronous callbacks need only implement a single line pattern before proceeding with a type safe result.
  • Lower cognitive load. A developer utilising a library written with the `guard` pattern does not need to learn how the library’s result type behaves.
  • Clarity. A `guard` statement appears to me to be more readable than a `case-switch`. This is subjective, of course.

What do you think? I am not a Swift expert. Am I missing something obvious? Why would you choose to use a result type over a `guard` statement?

Cover image – A bee harvests pollen from lavender on Thornleigh Farm

Originally posted at The Practical Dev

Asynchronous Object Initialisation in Swift

Baby birds, rockets, freshly roasted coffee beans, and … immutable objects. What do all these things have in common? I love them.

An immutable object is one that cannot change after it is initialised. It has no variable properties. This means that when using it in a program, my pea brain does not have to reason about the state of the object. It either exists, fully ready to complete its assigned duties, or it does not.

Asynchronous programming presents a challenge to immutable objects. If the creation of an object requires network I/O, then we will have to unblock execution after we have decided to create the object.

As an example, let’s consider the Transaction class inside Amatino Swift. Amatino is a double entry accounting API, and Amatino Swift allows macOS & iOS developers to build finance capabilities into their applications.

To allow developers to build rich user-interfaces, it is critical that Transaction operations be smoothly asynchronous. We can’t block rendering the interface while the Amatino API responds! To lower the cognitive load yielded by Amatino Swift, Transaction should be immutable.

We’ll use a simplified version of Transaction that only contains two properties: transactionTime and description. Let’s build it out from a simple synchronous case, to a full fledged asynchronous case.

class Transaction {
  let description: String
  let transactionTime: Date 
  init(description: String, transactionTime: Date) {
    self.description = description
    self.transactionTime = transactionTime

So far, so obvious. We can instantly initialise Transaction. In real life, Transaction is not initialised with piecemeal values, it is initialised from decoded JSON data received from an HTTP request. That JSON might look like this:

  "transaction_time": "2008-08",
  "description": "short Lehman Bros. stock"

And we can decode that JSON into our Transaction class like so:

/* Part of Transaction definition */
enum JSONObjectKeys: String, CodingKey {
  case txTime = "transaction_time"
  case description = "description"

init(from decoder: Decoder) throws {
  let container = try decoder.container(
    keyedBy: JSONObjectKeys.self
  description = try container.decode(
    forKey: .description
  let dateFormatter = DateFormatter()
  dateFormatter.dateFormat = "yyyy-MM" //...
  let rawTime = try container.decode(
    forKey: .txTime
  guard let txTime: Date = dateFormatter.date(
    from: rawTime
  ) else {
    throw Error
  transactionTime = txTime

Whoah! What just happened! We decoded a JSON object into an immutable Swift object. Nice! That was intense, so lets take a breather and look at a cute baby bird:

Break time is over! Back to it: Suppose at some point in our application, we want to create an instance of Transaction. Perhaps a user has tapped ‘save’ in an interface. Because the Amatino API is going to (depending on geography) take ~50ms to respond, we need to perform an asynchronous initialisation.

We can do this by giving our Transaction class a static method, like this one:

static func create(
  description: String,
  transactionTime: Date,
  callback: @escaping (Error?, Transaction?) -> Void
) throws {
  /* dummyHTTP() stands in for whatever HTTP request
     machinery you use to make an HTTP request. */
  dummyHTTP() { (data: Data?, error: Error?) in
    guard error == nil else { 
      callback(error, nil)
    guard dataToDecode: Data = data else {
      callback(Error(), nil)
    let transaction: Transaction
    guard transaction = JSONDecoder().decode(
      from: dateToDecode
    ) else {
      callback(Error(), nil)
    callback(nil, transaction)

This new Transaction.create() method follows these steps:

  1. Accepts the parameters of the new transaction, and a function to be called once that transaction is available, the callback(Error?:Transaction?). Because something might go wrong, this function might receive an error, (Error?) or it might receive a Transaction (Transaction?)
  2. Makes an HTTP request, receiving optional Data and Error in return, which are used in a closure. In this example, dummyHTTP() stands in for whatever machinery you use to make your HTTP requests. For example, check out Apple’s guide to making HTTP requests in Swift
  3. Looks for the presence of an error, or the absence of data and, if they are found, calls back with those errors: callback(error, nil)
  4. Attempts to decode a new instance of Transaction and, if successful, calls back with that transaction:callback(nil, transaction)

The end result? An immutable object. We don’t have to reason about whether or not it is fully initialised, it either exists or it does not. Consider an alternative, wherein the Transaction class tracks internal state:

class Transaction {
  var HTTPRequestInProgress: bool
  var hadError: Bool? = nil
  var description: String? = nil
  var transactionTime: Date? = nil

    description: String,
    transactionTime: Date,
    callback: (Error?, Transaction?) -> Void
  ) {
    HTTPRequestInProgress = true
    dummyHTTP() { data: Data?, error: Error? in 
       /* Look for errors, try decoding, set
          `hadError` as appropriate */
       HTTPRequestInProgress = false
       callback(nil, self)

Now we must reason about all sorts of new possibilities. Are we trying to utilise a Transaction that is not yet ready? Have we guarded against nil when utilising a Transaction that is ostensibly ready?  Down this path lies a jumble of guard statements, if-else clauses, and sad baby birdies.

Don’t make the baby birdies sad, asynchronously initialise immutable objects! 💕

Further Reading

– Hugh

Lessons from releasing a personal project as a commercial product

Aliens. It all begins with aliens. Rewind to San Francisco, and a game developer named Unknown Worlds.  Unknown Worlds is awesome.  We’re chilled out, but we create wonderful products. The games we make bring joy to millions of people around the world. The founders, Charlie and Max, are just the coolest and most inspirational blokes.

Before Unknown Worlds, I was at KPMG. A bean-counter, not a programmer. I couldn’t tell computers what to do. But now, making games, I was surrounded by people who could.

I was so inspired by Brian Cronin, Dushan Leska, Jonas Bötel,  Steve An, and others. They were gods. They would sit in a trance for days, occasionally typing incantations on their keyboards, and eventually show us some amazing new game feature. I was in awe.

Dushan would say to me: ‘Just automate something you do every day. It will be hard, you will have to learn a lot, but it will teach you how to write code‘. So I did.

I hold Dushan (mostly) responsible for this mess

At KPMG I spent a lot of time doing battle with Microsoft Excel.  There is nothing fundamentally wrong with Excel. The problem is that it is an extremely generalised tool, and the work we were doing was not generalised. Too much time was spent copying and pasting data, sanitising data, shuffling data by hand.

When I arrived at Unknown Worlds, I started monitoring our sales. I channeled my inner KPMG and created glorious spreadsheets with pretty graphs. It was an awfully manual process. So, on Dushan’s advice, I started automating it.

The process was agonisingly slow. I would devote time after work, on weekends, at lunches: I had no teacher. Once I got going though, I was hooked. Tasks that used to take us hours at KPMG evaporated in moments in the hands of the machine. I felt like a magician.

With great power comes great responsibility. Soon I was writing code in our games. I thought I was pretty damn clever. Some of the stuff I wrote was super cool, one feature even got me invited to speak at Game Developer’s Conference. But damn, most of it was hot garbage.

Working on Subnautica taught me that mediocre programmers are dangerous to the health of large projects. Also dangerous: Reaper Leviathans.

There is nothing more dangerous on a big software project than a mediocre programmer. We’re like a radioactive prairie dog on heat: Running around contaminating codebases with bugs, indecipherable intent, zero documentation, no testing, and poor security.

Eventually I learned enough to realise I needed to ban myself from our game’s codebases. I was desperate to be better: I wanted to be able to contribute to Unknown Worlds games in a sustainable, positive way. One day I read a recommendation: Create a personal project. A project you can sculpt over a long period of time, learning new skills and best practices as you go.

Channeling Dushan again, I decided to start an accounting software project. Accounting software gives me the shits. As I learned more about code, I realised that most accounting software is shit. And it’s near impossible to integrate the big accounting software packages into other software.

How many software startups can you fit in one frame?

Piece by piece, after hours, over weekends, and at any time a healthier person would take a holiday, I put together a beast I called Amatino. It was always supposed to be something small. A side project that I would use myself. Haha… ha. Oh dear.

Today Amatino is available to anyone. It’s a globally-distributed, high-performance, feature-rich accounting wet dream. You can actually subscribe to Amatino and real money will arrive in my bank account. That’s just fucking outrageous!

Still can’t believe this is a real screenshot

Even better, I’ve achieved my original goal. I feel comfortable digging around in code on Unknown Worlds games, and am no longer a dangerous liability to our code quality. I can finally do some of what I saw Max, Charlie, Dushan, Steve, Jonas and Brian doing all those years ago.

Along the way I picked up a few lessons.

Lesson 1: Do it

Creating your own product is utterly exhilarating and mind expanding. I’m about as artistic as an Ikea bar stool, but I imagine this is how artists feel when they make art. It just feels great.

Lesson 2: Keep your day job

Alright, maybe quit your day job if it doesn’t make you happy. But if you are happy, keep at it. Over the past years I’ve given Unknown Worlds 100% and more. Unknown Worlds makes me super happy. To build Amatino simultaneously, I had to develop discipline: Every night, every weekend, every holiday, code like hell.

Spend enough time around Max (L) and Charlie (R), the founders of Unknown Worlds, and you will be inspired to do cool stuff

There are many benefits. First, you don’t lose contact with your work mates. Charlie, Max, Scott, Brandt, and many others are constant inspirations to me. Second, you don’t have to worry about funding, because you have a job. Third, you are kept grounded.

I think if I didn’t spend all day making games, Amatino would have sent me insane. I would have lacked direction, and woke up not knowing what to do. Instead, I worked on making games, structured my day around Unknown Worlds, and devoted focused, intense energy to Amatino when possible.

Lesson 3: Your partner comes first

No matter how important a milestone is, or how deep in thought you are, or how good you think your ideas are, you drop everything for your partner. You lift up your partner, you encourage your partner, you support your partner. Every day, without fail, without exception.

This was a hard lesson to learn. It is the most important lesson.

Without Jessica, Amatino would not have happened. And it is precisely because she took me away from Amatino that she helped. The ritual of cooking for her, sharing meals with her, going on dates with her, doing household chores with her, listening attentively to her thoughts, concerns, and dreams. All these things take immense time, time you might wish to devote to your project instead.

You must not make that trade. It is a false economy. Your productivity will suffer, your health and emotional wellbeing will suffer. The energy you devote to your partner instead of your project will come back to you tenfold and more.

Don’t bore your partner to death by constantly talking about your project. Most importantly, don’t put off big life decisions because you think the time will be right after your project is released.

Don’t put off the big decisions!

Lesson 4: Eat well, exercise, and don’t get drunk

You all hear this enough elsewhere. You have a day job, a personal project, and perhaps a partner too: You cannot waste time recovering from the ingestion of cognitive impediments.  Any social value you get from being drunk is utterly dwarfed by the opportunity cost of brain-cells not functioning at peak efficiency.

Your mates might give you hell for this. Don’t worry, they will still love you in the long run.

Lesson 5: Ignore the framework brigade

I’m building a Dockerized cloud Node app with React-native frontend on GCP powered by the blockchain.” Don’t be those people. Learn from first principles. Start with abstract design thought, not a list of software for your ‘stack’. Don’t be afraid to build your own systems.

Reach for third-party dependencies judiciously and only where absolutely necessary. Learn by dabbling in languages where you need to allocate your own memory, while leveraging the speed boost that comes with those in which you don’t. Build computers. Tinker with them.

You will learn a lot from building, breaking, and upgrading your own computers. This one was maybe me taking it a bit too far

Hot tip: If your elevator pitch contains the brand name of a third party dependency, you are violating Lesson 5.

Lesson 6: Be humble

Maybe some people get ahead in life by being arrogant, self-assured dickheads. In fact, I am sure that is true. If you want to build and release a product, you need to check your ego at the door.

Suck in information from everyone and everything around you. Approach the world with unabridged, unhinged curiosity. Even when you don’t agree with someone, give them your undivided attention and listen, don’t talk. Consider their advice most especially if it conflicts with your own assumptions.

Good luck!