Here’s the short version: I’ve made a Creative Commons licensed version of the Galaxy Brain meme images. I’m releasing these images under the CC-0 license, which means you can use them in just about anything, including commercial works. I’ve made versions with both a masculine and feminine figure. More notes below!
We made these because we’re in the middle of writing Head First Swift, to be published by O’Reilly Media. The Head First series of books is intended to be a lighthearted and fun way to teach technical content, and my co-author, Paris, asked if we could add a galaxy brain meme in the book. Unfortunately, the nature of memes means that not only do we not have permission to publish the material in a book, it’s next to impossible to try and track down the rights-holders for those images. So, I made my own.
While I was making them, I made a couple of changes to the commonly-used formula. The initial image in most galaxy brain sequences shows an x-ray image with a picture of a scaled-down brain; I’ve always found this to be pretty ableist, especially since the meme is frequently deployed to make fun of people with bad opinions. Bad opinions are rarely the result of brain physiology. So, I opted to make the first image simply a non-glowing brain.
After posting the images on Twitter, I received several requests for a version of the images that didn’t just use the standard masculine model. Hot takes aren’t the exclusive domain of men, so I put together an additional set. (When I made these, I was glad that I didn’t make the first image use a small brain, because that would have made that image much more likely to be used in isolation as a sexist trope.)
Thanks for reading this far, and I hope these pictures bring you joy!
It’s a bit tricky to see where to begin with OpenAI’s (in)famous GPT-2 model. This blog post is our first in a small series about NLP. We hope it helps!
Our preferred way of installing and managing Python, particularly for machine learning tasks, is to use the Anaconda Environment.
⚠️ Anaconda’s environments don’t quite work like virtualenv, or other Python environment systems that you might be familiar with. They don’t store things in the location you specify, they store things in the system-wide Anaconda directory (e.g. on macOS in “/Users/USER/anaconda3/envs/”). Just remember that once you activate them, all commands are inside the environment.
Anaconda bundles a package manager, an environment manager, and a variety of other tools that make using and managing Python environments easier.
➡️ While inside the activated Conda environment you’re using, install the requirements specified inside the GPT-2 repository:
pip install --upgrade -r requirements.txt
➡️ Use the model downloading script that comes with the GPT-2 repository to download a pre-trained model:
python3 download_model.py 774M
⚠️ You can specify which of the various GPT-2 models you want to download. Above, we’re requesting a download of the 774 million parameter model, which is about 3.1 gigabytes. You can also request the smaller 124M or 355M models, or the much larger 1.5B model.
You might need to wait a little while as the script downloads the model. You’ll see something like this:
This will fire up the model, allowing you to enter some text. You’ll eventually (likely after some warnings that you can ignore about your CPU and the version of TensorFlow) see something like this:
Model prompt >>>
Enter some text and press return (we recommend only a sentence or two), then wait a bit and see what the model generates in response. This could take a while, depending on your computer’s capabilities.
We’ll be back with a follow-up article, exploring how you can actually use GPT-2 for something useful. Stay tuned!
This is the first post in a two-part series. Stay tuned for the second part.
One of the projects that we’re most excited about is our ongoing work to bring Night in the Woods to iOS. This has been an enormous amount of technical effort, and has led to a bunch of very cool spin-off projects – for example, Yarn Spinner only exists because we built it for Night in the Woods, and the research we’ve been doing in sprite compression to fit the game into a tiny amount of memory has been a blast.
One of the things we’re doing right now is improving the performance of the game on the device. Just like when you’re building a game for PCs or consoles, you want your game to be running at a high, stable frame rate. For mobile devices, this is a little more complex – you also have to consider the impact that the game has on the phone’s battery, because nobody wants to play a game and then find that their phone’s about to die.
Another important element of performance for mobile games is thermal impact. When the phone’s running a game, it’s making the hardware do a lot of work, and this makes it heat up. Modern systems – that is, anything made in the last 40 years – detect when they’re under thermal stress, and reduce their performance to avoid damage. This means that you might be able to achieve a solid 60 frames per second when you start playing, but that might get laggy in a couple minutes (or seconds!) of play. (Thermal stress is also a consideration for PCs and consoles, but it’s more critical for phones due to the reduced amount of space inside the chassis, the lack of active cooling, and the fact that you’re holding it in your hands.)
Spotting the Problem
The first thing that we noticed was that the reported frames per second in the game was low.
(We use an FPS counter called Graphy to visualise FPS. Graphy’s great – it’s easy to set up, reliable, low-impact, and open source. You should use it!)
The FPS counter was reporting an FPS count of about 20 to 40 frames per second on my iPhone X, depending on the scene. The other thing that was spotted was that the frame rate was inconsistent to boot.
What was going on? Night in the Woods, despite its gorgeous visual style, doesn’t actually have hugely complex scenes. When diagnosing a problem like this, the first thing to do is to figure out where the bottleneck is – on the CPU, or the GPU.
Finding the Bottleneck
Xcode has a very simple tool for checking which part of the system is under the heaviest load. When running the game from Xcode, you can just click on the Debug navigator, which will show you a summary of the app’s performance.
The CPU usage and energy impact here are pretty high. Not enormously high, considering it’s a game, but still high. As the game itself reports, it’s barely achieving 30FPS frame rate. When we select the FPS element, we get some more detail:
The FPS report is showing that the device is 100% utilised, and that the CPU and GPU are both taking a full 28 milliseconds to produce the frames. The renderer is under significantly more load than the tiler, which effectively means that most of the work is being done by the fragment shaders, and not by having to deal with a lot of geometry (something that we’d expect, given that the polygon count of Night in the Woods, like most other 2D games, is not terribly high.)
Even though the CPU and GPU times were the same, this doesn’t necessarily mean that they’re under the same degree of load. Switching over to the CPU report showed that the CPU wasn’t really at maximum load (though this can be deceiving, since it may have been the case that a single core was at max load.)
The analysis that we’d seen so far seemed to suggest that the GPU might be the best place to look at, so we pulled out one of the two biggest and baddest tools in the chest: Instruments.
Instruments can show a huge amount of data about the performance of an app. Happily, recent versions of the app come with the Game Performance template, which sets up a number of recorders that relate to how a game performs.
So, we ran our scene while recording data, and got back a simply enormous amount of data. It’s actually rather pretty.
There’s a lot of charts here, but the key thing to look at is at the bottom of the window, where the GPU state and display information is shown. Let’s zoom in on that.
The purple bar at the top shows when the GPU was busy doing something. The bar is entirely filled, which means that there was no point in the run when the GPU was allowed to be idle. That’s bad, because when the GPU is active, it’s pulling energy out of the battery, and heating up the phone. We already knew this, because Xcode’s report was showing us that the device utilisation was 100%, but it’s nice to see this in a little more detail.
Where we start to see more useful information is in the ‘Display’ instrument. The Display instrument shows four charts:
The current frame shown on the screen
When a new frame was delivered to the screen, ready to be shown
When the screen vsync’ed (that is, when it attempted to swap to the next available frame)
Any frame stutters that Instruments found. There aren’t any in this screenshot, but that doesn’t mean that the display isn’t stuttering.
Take a close look at the top row of the Displays instrument. It’s showing the duration of each frame on screen, and they’re not all the same. Some are 33 milliseconds, some are 16. That’s what’s causing the janky appearance.
What’s interesting about this is that the frames are taking the same amount of time to be produced! We know this because the second row, labelled ‘scaler’, shows that each frame is being submitted to the GPU after about the same amount of time. The reason why some frames list for 33 milliseconds and some last for 16 is due to the fixed rate at which the display is swapping to the next available frame.
Understanding Frame Judder
To figure this out, let’s look at the timing information for four frames.
Here’s what’s happening at each of these four steps.
The blue frame is on screen. The next frame, shown in green, is submitted to the display.
The display hits its next vsync point. The green frame is ready, so it’s shown to the user.
The orange frame is submitted to the screen, but it’s just too late for vsync. The screen therefore has no new frame to show, so it keeps showing the green frame for another sync interval. The orange frame is eventually shown at the next vsync.
Meanwhile, the blue frame was being drawn, and it’s ready to go right away. It’s submitted to the display, and drawn right after that. The orange frame was only on screen for a single sync interval.
The problem here is not that the frames aren’t taking too long to draw. The problem is that the game is trying to send them to the screen at too high a rate. Ironically, in order to make the game smoother, we need to reduce the frame rate.
This was a one line change:
Application.targetFrameRate = 30;
The result was this:
There are two things to note here.
First, the purple area now has gaps in it, indicating periods of time when the GPU was not active. This is a good thing! Less energy being drawn from the battery, and less heat.
Secondly, all of the frames are on screen for a consistent amount of time. They’re never missing a vsync, because the game isn’t trying to get them onto the screen as fast as it can. The game can now focus on operating at a consistent 30 frames per second.
The result is lower power consumption, and a smoother gameplay experience.
We’re Not Done Yet
But this was only part of the problem. As I mentioned earlier, the scenes in Night in the Woods are not incredibly complex; why was it taking so long to produce the frames? To solve that question, we needed to take a much closer look at the internals of how the GPU was drawing each frame.
We’ll look at how we achieved even better results in the next post. In the meantime, I’ll tease you with this: