Learning to Use Unity to create an AR application

After learning how to use Unreal Engine 4 and understanding the pros and cons of the software, I decided to jump over to Unity.

Like Unreal Engine, Unity is a licensed game engine used to develop video games and other applications. The engine is a real-time development platform and used to create 3D, 2D, VR and AR mobile and low-end PC and console games, as well as simulations and other experiences that can be used as both tools and entertainment. (Takahashi, 2018).

Unity makes their software accessible and easy for anyone to use, even if they have a lack of technical knowledge. This is because Unity has a user-friendly interface that's easy to handle. Furthermore, you do not need extensive programming knowledge or experience as very little of the game development process on Unity is about coding. This means developers can focus on creating content and the story instead. Perhaps this is why in recent years there has been a surge in story-focused games with cinematic journeys and visuals. "Art might become most games’ primary identity." (Axon, 2016). Unity is also free for any personal use. Furthermore, for company's with annual gross revenue of $100,000 and over, they only need to pay an upfront cost of $1,500 per Professional Edition license instead of conditional royalties, like Unreal (Rao, 2015). This means, that for the most part, Unity will be cheaper to use than Unreal. According to the Unity website in 2018, "more than 6 million registered developers use the platform, and 770 million gamers enjoy Unity-made titles." (Axon, 2016). This increased accessibility allows for a richer diversity among developers and therefore projects - reaching wider audiences and creating niche markets that are able to please anyone and everyone.

“The idea of even making a game in 3D in like two or three days...used to be basically something you would only be able to do if you had a 10-person triple-A team who knew exactly what they're doing...Now you see like a one-person team or two-person team just make a complete, fun game in 3D in a single weekend. " - Matt Rix, co-founder of Milkbag Games (Axon, 2016)

However, this increased accessibility does have it's limitations:

some professionals argue that due to its accessibility, the gaming industry is being compromised. The overall quality of games is dropping as people with less and less professional expertise and creative insight are creating content (Axon, 2016). This also means that the number of games being released is increasing, especially in copy cat games that are lower in quality and market themselves at much lower prices (Galyonkin, 2020). In addition to this, with so many games being developed using Unity, they all start to feel very similar - this is no surprise when a lot of them use the same graphic design, physics calculations, and other foundational script that Unity offers. All of this threatens the stability of the careers and lives of those trying to earn a living through game development (Wright, 2018)

There is nothing stopping developers from customizing the engine to create their own unique games and experiences. However, this would require the developer to have a deeper understanding of C# and JavaScript, which a large percentage of indie game developers who are making their first game using Unity do not have, simply because they are coming from a very amateur background. On the other hand, due to a large library of accessible tutorials, it is not unfeasible for amateur developers to learn how to code and programme for Unity.

The "Indiepocolypse" refers to the overcrowding of video games on the market and the effect that has had on indie games and those that develop them. Jordi de Paco, a Spanish video game developer and founder of Deconstructeam, says that when his studio released 'Gods Will Be Watching' in 2014, a point and click adventure game, they made a profit just from the pre-orders alone. However, their 'The Red Strings Club', a 2018 cyberpunk adventure game, only started to make a profit after months of it being released. He goes on to suggest that "If we released 'Gods Will Be Watching' today, it would’ve failed...because it used to be 20 games a week on Steam, and now it’s 20 games a day.” (Wright, 2018). According to SteamSpy, the total number of games released in 2013 was 569, so far, in 2020 there have been as many as 9516 games released (Galyonkin, 2020).

“Everyone can make games, but be realistic. ... It used to be that you could do something that nobody had ever seen before, or you could do something familiar really well. Now, it has to be innovative and have incredible quality.” -Jordi de Paco (Wright, 2018)

It would be unfair, however, to blame Unity, alone, for the increase of saturation of the market, especially when it has opened so many doors for developers. With its extended cross-platform support, the integrated development editor that makes coding easier, detailed documentation that explains every small topic, a huge assets store and the ability to debug any issues at runtime. Because of all these benefits, Unity has a large and active community, allowing developers to reach out to one another to collaborate or help solve problems. It is also why the engine has been adopted by industries outside video gaming, such as film, automotive, architecture, engineering and construction (Wilfred, 2020).

On the other hand, there are some aspects of Unity that aren't as advanced or user friendly as Unreal Engine, the graphics, for example, Unreal is much more photoreal than Unity. Unity is also a lot slower at rendering a project. Unreal Engine uses visual scripting with blueprints and nodes, making it, arguable, more accessible than Unity for people with a lack of programming experience. Furthermore, in my opinion, the asset store for Unity is not as good as the asset store for Unreal. Nevertheless, Unreal is not suited for smaller games and applications. Unity is much better and faster at developing mobile games, therefore I will be progressing on with Unity as I intend to create an AR app that audiences can use to be able to interact in an immersive theatre space.

Augmented Reality is bridging the gap between reality and the virtual world by allowing people to interact, in real-time, with people and objects that are projected into a space using a device such as a phone. It is still very much at an early stage, needing much development, however, it has already made headway in many industries, from marketing to medicine.

Magic leap, a company focused on developing spatial computing have released a headset that superimposes 3D computer-generated imagery over real-world objects and spaces which "projects a digital light field into the user's eye"(Huet, 2015). The founder and CEO of Magic Leap believe that with this head-mounted virtual retinal display, we will one day no longer have a need for physical screens. However, we are still far off from getting to that point. At $2,295, It's currently too expensive, it's smaller than a VR headset, but it's still too clunky for prolonged use and it lacks content. It isn't consumer-ready yet. (Takashashi, 2018)

The potential of Magic Leap and other spatial computing companies is exciting, and I can't wait to see what it does for theatre and TV. AR is already starting to be used on programmes like BBC's Strictly Come Dancing to add context to the world of each dance. I want to use AR to give audiences a new way into a narrative, to allow them to interact within a space and discover things for themselves, thus giving them an increased sense of ownership over the story. I want to do this through the use of smartphones, as, in the 21st century, we all use our phones to connect with the world around us. We use it to learn and to teach others, to reach out and re-connect, spreading love and empowerment, however, we also use it to spy and gossip, to incite self-hate and lies. It's an incredibly powerful tool, and it would be foolish not to use it. It offers my audience a familiar context and leaves them free to explore the wonders of an immersive and interactive experience.

"We have to realize that all people are walking around with a smartphone in their pocket... it's become the most important screen in the world, so the step to use that ingredient to allow interactions with the display is just a very small step." - Frederik De Wachter, (Digital Signage Today, 2016)

The first thing I made on Unity was a landscape as it was the most recent project I had done on UE4 and therefore I would be able to make a direct comparison of the two programs. This also allowed me to get to grips with the basic applications and tools so that I could go on to design more complicated applications and focus in on what it was that I wanted to get out of unity - AR.

Due to the fact that I was already comfortable with Unreal Engine 4 and using blueprints, it took me a little while to get used to using Unity as it requires actual coding. However, it wasn't long before I was happily exploring the software and playing around with the different things I can create, and it soon revealed itself to be a lot less complicated than Unreal. I will say that I did prefer building an environment in Unreal, I enjoy using nodes, as opposed to traditionally written code, and I like how much more specific it allows me to be. Furthermore, the end result ends up being a lot more photorealistic. For lack of a better comparison, using Unity feels like playing one of the early Sims games; most of it is pretty self-explanatory, its simple, fun and easy to jump right in, making content development incredibly accessible - I didn't have to look up basics like how to navigate the camera. In comparison, UE4 is more like Sims 4, requiring more time and attention; I needed a lot more external support when learning how to use it and a lot more powerful hardware. However, at the end of the day it creates a richer, more customisable and more detailed product. However, Unity is much better suited for smaller applications like mobile apps. Therefore, when building my AR app, Unity will be a much better software to use. In contrast, when building the environment for my project, Unreal is the superior software.

As previously stated in another one of my blogs, I am interested in how I can get audiences to experience memory through AR - I have looked at volumetric capture and imagery, however, 3D video processing of volumetric video is very expensive and the data ends up being very heavy, which leads to glitches and slow run time. Because of this, I'm going to start with 2D image projection. This will allow for a better flow as the data won't be so heavy, which will help keep the experience immersive and avoid any opportunities for the audience to be taken out of the narrative.

I felt a lot of achievement when I had successfully created this prototype which presents two types of AR content: 1) a replacement image that swaps an existing photo with another, and 2) a video overlay that turns a static photograph into a moving video clip. During the development process, I came across a lot of obstacles - for a while I could not get Vuforia to cooperate with Unity. To begin with, I could not get Unity to even import Vuforia assets. However, after reading through some forums I found that older versions of Unity downloaded with Vuforia already embedded into the software, so I downloaded the Unity 2018.4.30f1 version. My biggest problem, however, was that I could not get my video to play, no matter how much I played around with the settings in Unity I kept getting white noise. After a lot of digging, I came across a tutorial that took me step by step, showing me how I can edit the C# scripting for Default Trackable Event Handler (script) and create a new class with ITrackableEventHandler to allow my video to play.

I have continued to play with Unity and Vuforia to create AR content, experimenting with 3D character models and getting them to do things like dance or kiss. It was very easy to do due to the help of Mixamo, a 3D character animation web service, however, I was disappointed to find that Adobe Fuse had been discontinued as it meant I could not design my own characters and I had to use ready-made templates. However, I will continue to look for other software that will allow me to animate my own creations. I also want to work on producing live-action performances through AR.

Furthermore, I have been working on adding audio to my 3D models, but I am struggling with the script again. The problem at the moment is that I am copying other people's script that I am finding online ( using GitHub) without fully understanding what it does. Therefore when I come across any errors, I am ill-equipped to fix them (though not for a lack of trying) and therefore unable to add the script components. Furthermore, due to the fact that I have different hardware to those doing the tutorials, and often can't confirm if I even have the same version of the software, I am unable to follow the tutorial step by step exactly. This overall makes for an incredibly frustrating situation. I have decided that I need to strip back and actually look into online courses that will teach me C# script so that I can understand the language more and become more confident in writing my own code and solving problems.

Axon, S. (2016). 'Unity at 10: For better—or worse—game development has never been easier' [online] Ars Technica. Available at: <https://arstechnica.com/gaming/2016/09/unity-at-10-for-better-or-worse-game-development-has-never-been-easier/ >[Accessed on 10th December 2020]

Digital Signage Today (2016). 'Making the Customer Experience Interactive', Digital Signage Today [online]. <https://www.digitalsignagetoday.com/blogs/making-the-customer-experience-interactive/?printable=true> [Accessed 30th October 2020].

Galyonkin, S. (2020). 'Monthly Summaries -Games Released In Previous Months' [online] SteamSpy. Available at: <https://steamspy.com/year/> [Accessed 15 December 2020].

Huet, E. (2015). 'Magic Leap CEO: Augmented Reality Could Replace Smartphones' [online] Forbes. Available at: <https://www.forbes.com/sites/ellenhuet/2015/02/24/magic-leap-ceo-augmented-reality-could-replace-smartphones/> [Accessed on 15th December 2020]

Lapowsky, I. (2015). 'Magic Leap CEO Teases 'Golden Tickets' for Its Augmented-Reality Device' [online] Wired. Available at: <https://www.wired.com/2015/02/magic-leap-reddit/> [Accessed 15th December 2020]

Rao, A. (2015). 'An Indie’s Guide to Costs and Scripting with Unity 5 and Unreal Engine 4' [online] Gamasutra. Available at: <https://www.gamasutra.com/blogs/ArjunRao/20151123/260140/An_Indies_Guide_to_Costs_and_Scripting_with_Unity_5_and_Unreal_Engine_4.php#:~:text=Unreal%20Engine%20usage%20includes%20royalties,annual%20gross%20revenue%20exceeds%20%24100%2C000> [Accessed on 11th December 2020]

Takahashi, D. (2018). 'John Riccitiello Q&A: How Unity CEO views Epic's Fortnite success'. VentureBeat. [online] <https://venturebeat.com/2018/09/15/john-riccitiello-interview-how-unity-ceo-views-epics-fortnite-success/> [Accessed on 10th December 2020]

Wilfred, T. (2020) 'What is the best game development engine?' [online] Dev Community. Available at: <https://dev.to/thomaswilfred15/what-is-the-best-game-development-engine-2e47> [Accessed on 10th December 2020]

Wright, S. (2018) 'There are too many video games. What now?' [online] Polygon. Available at: <https://www.polygon.com/2018/9/28/17911372/there-are-too-many-video-games-what-now-indiepocalypse> [Accessed on 10th December 2020]