Visual scripting is still programming
There are more and more ways to learn how to program, and there are a lot of tutorials on the internet. Yet, we still have the issue everywhere of companies wanting to hire technical artists, technical designers, technical narrative designers, etc., and not being able to find them. There is so much out there, yet the barrier still exists.
Yet at the same time, we have seen a wave of visual scripting appear. Noodle graphs of ‘code’ and ‘programming’, that potentially allows ‘anyone’ to be able to make ‘anything’! The idea is that those technical folks can all do things they need themselves, and not need a programmer for support. But they are essentially still programmers, with similar issues programmers have to deal with, only now working visually. Those folks are still hard to find, expensive to hire, and they have to learn your particular editor, system, and tools anyway, so the leadup time for their efforts can be long too.
With the right visual scripting tool in the right editor, you can make anything you need for any specific project you have. The possibilities are endless! You can reach any result you need! None of those platitudes mean much. You could give someone a terminal and assembly code, and they could potentially make any kind of game or software. That is technically possible. The possibility existing is very different from it being easy, fun, nice, and comfortable to do.
Over the years I keep seeing folks say “Make a game without having to learn how to program!” or “Make a game with no code!” and then pointing at visual scripting interfaces, and node graph editors, as if they are not also filled with workflows that have a big Venn diagram overlap with programming skills. You have probably seen these interfaces before. Unreal has Blueprints, Unity has Bolt and Playmaker, and many other editors have other versions of visual scripting and node graphs. These node graphs require you to put the right nodes in the right place, and then wire them up correctly. That’s all you need to do. So, what is the difference between programming and visual scripting?
Programming vs Visual Scripting
It used to be that you had to type out specific code to get your required end results. You would have to know exactly what to type out, and how it would have to work and interact with your other code. What would start a function? How would you close one? How is a value defined? What needs to be bracketed, and what needs to be in those brackets? You have to type it all out one character at a time. Nowadays we have more and more autocomplete features, so there is less guess work, and less typing, but in many cases it is still unclear for folks what they need to write down for the end result they want. Especially if they switch between programming languages or APIs.
In comes visual scripting: You have to line up specific nodes to get your required end results. You do have to know what nodes to use, and how they would have to work and interact with your other nodes. What would trigger the functions you need? How would you test it out to see if it worked? How is a value defined? What needs to be connected to each other, and how should it be connected? You have to line it all up, one node at a time. There is autocomplete for the available nodes, so there is less guess work, and less typing, but in many cases it’s still unclear what nodes exactly need to be lined up. Especially if you switch between editors or visual scripting languages.
A neat example of this is for loops in code, and mathematical notations. Here is a tweet from Freya Holmer: On the right is code, on the other is a single visual. You can kind of see it like a node, can’t you?
btw these large scary math symbols are just for-loops pic.twitter.com/Kq6dcihPp4
— Freya Holmér (@FreyaHolmer) September 11, 2021
So, from a pure workflow level, there really isn’t that much of a difference in the user experience, even if under the hood it is very different from a technical perspective. It has become visual, so you have boxes and lines, but the knowledge you need to line those up and work with them is extremely similar. You face the same issues when working in one or the other. You can create a huge mess of spaghetti with nodes, and you can create a huge mess of files, functions, and references using code. You can make very simple and clearly commented node graphs, and you can make very simple and clearly commented code.
Visual scripting still has the same issues as regular programming. For example: Which 6 nodes, out of the 20000 available, do I need to put in the exact right order to get the end result I want? You end up still having to know exactly what to do. You’re not typing it out, as now you’re visually laying nodes out, but is visually trying to find the node you work with that much different than textually trying to find the function you need to work with? Autocompleting to the textual name of a function in code is exactly the same as autocompleting to the name of a node in a graph.
Other examples I often hear brought up are Blockly or Scratch.. It allows users to set up code in a visual way by use pre-defined parts, and it looks like this:
You are dragging and dropping the pieces on the left together to automatically generate the code on the right. So what is really happening here from a user experience point of view? You are essentially copy and pasting much faster. That’s it! It is a much nicer visual interface for quickly copy and pasting templated sections of the code. It helps folks to more easily learn how code is set up, and what parts of it are necessary, which is great and really helpful. And, it is still programming. These systems of predefined blocks to set up logic is similar to a lot of visual scripting and node graph editors, where instead of copy and pasting sections of code, you are dragging nodes around, and reconnecting them with wires. It is quicker, and parts only fit where they are allowed to fit, which is helpful, but you still have to know and understand the logic behind it to do anything useful.
When it comes to debugging, the same issues between coding and visual scripting still occur too. Did you forget to close a parentheses, or forgot to add a semicolon? Your debugger may give you a warning of a function or wrong bracket placement way further down than where you forgot to place it, as technically all the code before that can still run. The same goes for visual scripting tools: Did you forget a value or node somewhere? Your visual scripting debugger may give you an issue of a function or value in a completely different place, as all the other nodes before it may still work fine. You will still have to deal with all kinds of non-warning and non-error issues, of characters, vehicles, objects, etc, behaving in different ways than you expected, due to a particular setup of nodes missing something, just like you would have when working with code.
Visual scripting can be much easier though, with many templates that are easy to use for anyone to set up. (Though the word ‘template’ is a bit overused and can mean many things, so you may know them as ‘compound nodes’ instead, or some other name). For example, let’s say you have a node for ‘Screenshake’, instead of having to put 20 nodes together. With those 20 nodes you may have to calculate relative vectors of the world and the camera, do a whole bunch of math, etc, to get a screenshake to occur. Is that any different than writing your own screenshake function in code versus a ScreenShake() function existing in the API you are using?
When it comes to how to learn how to code, or how to visual script, the same issues exist. How do you know what is available, and what you can and cannot do? Will you read all the documentation and watch all the tutorial videos to find the one thing you need? Will you use google to find a solution, because the search system on the documentation website is not accurate enough?
You see this with folks who switch from Unreal to Unity, or from Unity to Unreal. Some of these people have incredible skills, sold millions in product, and yet they cannot quickly and easily make the things that they do know how to make in another editor, with another visual scripting tool. It’s like switching programming languages, where you may be a professional in one, but a novice in another. Even though you want to build the same thing in both, and you have proven your expertise doing it in one of them, you still have to relearn how to work with it.
Both Unreal and Unity, as well as many others over the years, can boast ‘You can do anything!’ which while technically true, is an empty boast. How simple and easy is it to do a ‘basic’ thing? How fast can a new user use these tools? Those are much more important metrics to look at. The same goes for when someone says: 'Well, once you get used to it, it's brilliant!'. That is also a totally empty boast, because it can be applied to any tool anywhere. And it has. If you see an expert using a tool, it will always seem magical. You will wonder "How did they do that so fast?! What hotkey was that?!". An expert using Maya will make Maya seem magic. An expert using Houdini will make Houdini seem magic. An expert using After Effects will make After Effects seem magic. That does not mean the user experience is necessarily great.
It is a much more brilliant thing to be able to say "Anyone can pick this tool up in 30 minutes, without a video tutorial or documentation, and create something they want to. Then when they become an expert by experience or learning, they can make their usage of the tool look like magic." If you only have the ‘magic’ and ‘You can do anything’ bits for your tool, you have only reached the status quo we have reached for many tools out there in the last 30 years. Which is super rough to deal with as a developer, because of the sheer amount of effort and time to get to the point where you are even hitting the status quo, and where it can be magical for an expert. It is like you have reached the top of the mountain, loudly cheer in victory, and only then realize you still have halfway to go.
Let’s grab an example of something that should be simple: How do I get a door in my game?
When is a door not a door?
On the Unreal Engine documentation pages is a tutorial about how to create a door, found here.
This should be an exceedingly simple door: It opens when the player gets close. That should be an easy thing to do, right? There should be some kind of distance to the door, and when the player is in that distance of the door, it rotates 90 degrees.
The end result blueprint of that tutorial is the image below:
Without following the tutorial, and without going into depth, do you understand what this blueprint is doing? Probably, yes! The nodes have clear names. Now: How would someone who has not worked with this visual scripting system know how to grab these exact 6 nodes out of the thousands available, and string them up exactly in this way to get the correct end result? Would you really search for a ‘Make Rotator’ node instead of trying to find the rotation value of the door object and trying to rotate it through there?
And of course, it isn’t like this is a surefire door for most video games cases. As already clearly indicated by Liz England’s ‘The Door Problem’ in 2014. How does this react when there is an object stuck in the door? Or the player? Or an ally of the player? What about a quest item on the other side of the door? Can this door be hacked, or does it need a keycard? All of these things can be added to that door. It is all possible, with visual scripting!
Do you know how to do all that? Do you still have to dig through documentation, lists of nodes, edge cases, etc, to find out how to do all of that? Then you’re still programming. It has become visual, and arguably nicer to look at, but the original and eventual workflow issues still exist. Those issues have simply been shifted into a visual interface. It looks nicer, but very little has been resolved from a workflow point of view.
Visual scripting has helped many folks get into programming. It looks nicer, the overview of it is easier than code (depending on how split up all their files are, and if there is not a spaghetti of lines all over the place.), and in many cases it’s easier to explain too. Visual scripting is a step forward. We cannot, however, think of it as an end goal by simply having made programming more visual. Regular IDEs have done great advanced with making code more visual too, with for example adding minimaps so that it is easier to find, recognize, and spatially orient yourself in huge files of code, or by allowing you to click on a function and seeing where it originates right away instead of having to manually use a search function, etc
Visual scripting existing, and ‘everything being possible’ through visual scripting does not resolve the issues that are still present and reminiscent of regular code programming. So, what do we need to look to instead?
What is the end goal?
It is very similar to how APIs improve over the years: By making it simpler and simpler to do basic things. A function for screenshake, instead of having to math one up yourself. A function for inventory management, instead of having to program yet another one. Such as it goes for code functions, such does it go for visual scripting systems: Is there a singular node that allows a user to do the thing they want to do?
And we have already seen the end results when you take this many steps further. You are reading from a website built on such a system: Squarespace.
To build a website nowadays there are many providers that allow you to work entirely without code. You drag and drop elements on a page, and it generates all the HTML, CSS, etc, outside of your view. You never have to see it if you do not want to, but you can if you would like to. There are some limitations, as any language or API has, but it allows you to set things up truly without code. Truly without scripting. Truly without visual scripting. Without programming. Because visual scripting is still programming.
So the end goal we need to shoot for is cutting the cruft and making it that much easier for anyone to create anything they want. To make higher level functions in APIs, and make higher level functions in visual scripting languages. For example, if you want an object to directly face another object: Should that require you to understand math, vectors, and a whole bunch of nodes? Or should there be a simple node that says: “Make object A face object B over the X axis.”?
Now you may say “Yes, but my needs are super custom! I need really specific things in really specific ways that the templated nodes definitely cannot account for!”. Really? If you want a healthbar on an NPC, shouldn’t that be a simple node everyone can use, with some values to customise it? Why should anyone have to connect a whole bunch of nodes together just to have a bar appear that is always visible next to a character, and slowly shrinks and goes from green, to orange, to red depending on some health value? The visuals may be particular to your game, but the way it reacts should be very similar to other games. That is a huge hallmark of a good user experience: To have your experience have similar elements as other experiences, so that they are easy to understand and use. Customising things infinitely is neat, but just because you can do ‘anything’ with visual scripting does not mean it has resolved really basic user experience problems of being able to do basic things simply and fast. And you can have both: Templates & compound nodes can exist for basic game necessities, while at the same time nodes can exist in which users can type literal code to do custom work if necessary. There is no need between a choice of one or the other. You do not have to pick between a good user experience, or powerful tools. You can have both, if you plan for it before you start building, and you have engineers and UX designers working hand in hand.
But, wait, visual ‘scripting’?
Look, the whole argument of what the difference is between scripting and programming is incredibly tedious. Is something programming because it is lower level? That depends on the API. C++ can be high level depending on the API you are using. So what then? Is scripting different because it has a faster iteration time or hot reloads? If so, what is live C++ then? Do we suddenly say that coding C++ is ‘scripting’ too if you are using that? Or is the difference between programming and scripting that you make the whole program with programming, and only smaller bits with scripting? If so, what if you’re using Unreal and only ‘scripting’ the missing functionality you need in C++ for your game to work? Does it then suddenly become scripting?
It is a tedious and unending argument to have, as the difference is what you think the difference is. Fact does not come into play, because if it would, we would all realize the argument is too silly to have. In the end, what really matters is the user experience of those creating content, and what they are able to build for others to experience. Whatever you call it is secondary to that.
Conclusion
Having ‘visual scripting’ in your editor does not magically make your user experience better, and does not resolve your needs for technical artists, technical designers, etc. The visual scripting API has to have easy to use templates in it that allow anyone to make basic gameplay and functionality without having to have any technical knowledge. And knowing which nodes out of the thousands to use, and how to line them up, is technical knowledge.
To be clear here: I am not saying there should only be templates in visual scripting systems. There are varying levels of depth, as with any language or API. You have some higher level functions, and some lower levels ones. The neat thing is that in node graphs editors, you can sometimes dig into a node and change its functionality. Maybe even dig further into a node that is in there, and go multiple levels deep, to then customize it and create the perfect functionality for you. All of that is possible, and necessary, if we really want folks to be able to build projects ‘without any programming experience’.
So, what can we learn from this?
Visual scripting and programming can be exactly the same in their eventual user experience, depending on the language, API, and user interface.
To really make ‘no code’ possible within game editors, there have to be many more templates. Just as with Squarespace, other website builders, etc.
Adding a ‘node graph editor’ or ‘visual scripting language’ does not automatically lower the technical knowledge barrier for your users, and is not a magic bullet savior for your tools. You may just be creating a nicer and more visual way of programming. Which is nice! It just does not mean ‘No code’ or ‘No technical knowledge’, and your users will still have to have technical skills, which are expensive and hard to find.
Thank you for reading! Thank you as well to David Lightbown for feedback and proofreading. If you enjoyed this post, please subscribe below to get updates when a new post is published. Also, this article is an expanded section from a talk I gave at the Gamescom Congress in 2022, which you can watch here.