Reflective Commentary

This module has been a challenging and rewarding process. To continue to grow as a developer and creative it is important that I outline key reflective commentary demonstrating an in-depth understanding of the production process, review my research methods and critically evaluate my completed artefact.

Contextual Summary

Before I started this MA I had very little experience in programming and had little to no knowledge about immersive technology. I came from a performance background with the intention to learn new ways to support live events and performances with immersive technology and a key interest in user experience. Live theatre is at risk of being marginalised due to government funding cuts (Bakare, 2019), outdated business models (Culture Track, 2017), the rise in ticket prices (TRG, 2019) paired with an ageing audience (Snow, 2016) and the development of digital entertainment (Ofcom, 2020). Therefore it is important that theatre venues and organisations adapt to modern trends in order to stay relevant. Such as creating accessible experiences that can be accessed in the comfort of the participants home; or experiences that incorporate new and exciting technology that allows audiences to control the narrative and in consequence broadening representation and attracting new demographics.

Theatre and other live experiences play an important role within our society - with the creative arts contributing to a significant portion of our economy. As part of the Department for Digital, Culture, Media & Sport, the creative arts industry has had a faster Gross Value Added (GVA) growth than the UK economy on average since 2011 (DCMS, 2021). Furthermore, studies such as the ones carried out by Encore Tickets demonstrate how theatre can improve social tolerance, academic performance, promote positive social change and support mental health (Kirkham, 2019; Richardson, 2019). Such findings highlight the importance of preserving theatre’s relevance within modern entertainment and education.

My aim was to produce an artefact that incorporated live storytelling with interactive technology, creating an intergenerational experience that attracts both younger and older audiences as well as encourages those who wouldn't normally go to the theatre to participate in live storytelling experiences. This required research into different types of interactive and immersive video games: looking at what makes them successful, where their limitations lie and how they are already influencing theatre. I have been able to use the medium of a blog to explore critical theory and case studies that have broadened my understanding, help me decide who I want my target audience to be, and develop a story that is rich and accessible whilst being flexible enough for audience interaction.

One case study that offered a lot of insight was of a theatre company called Creation Theatre, who wrote a Digital Toolkit as a response to the restrictions around COVID-19. As they adapted their business model to create more digital work they found that they were able to reach wider audiences on an international level and increase the accessibility of their shows, creating opportunities for d/Deaf, disabled and neurodiverse audiences, families and elderly audience members (MTM, 2018). Digital experiences allow organisations to include subtitles and captions, audio descriptions, and loop systems; and foster relaxed environments. All this contributes to removing a lot of the pressures and stigmas that can come with traditional theatre, and thus, contribute to the recruitment of new, younger and more diverse demographics.

Within the Toolkit, Creation Theatre elaborate how they were able to create personal experiences for their audience, offering advice on what worked and what didn’t: “involve your audience in the story, they should feel a part of the progress of the play” (MTM, 2018). By putting the audience at the heart of the narrative, I am able to inspire and entertain them whilst facilitating an environment in which my audience can make personal connections with both the narrative and with each other, enriching the whole experience.

To help me achieve my goal of creating an emotionally complex videogame type experience I got in touch with Gazooky Studios, an organisation that specialises in making mixed reality story-based content, and together we produced a storytelling experience that celebrates the history of St George’s Hall and utilises interactive projection mapping, AR, film and spacial audio. I felt very lucky to be a part of such a fantastic organisation that was willing to facilitate my own exploration; however, my time with Gazooky presented a steep learning curve with many challenges. I had to learn how to work effectively for a client, respond to feedback and work around the limitations of the technology available.

Preliminary Research

The early stages of my process included going to the Liverpool Record Office to inspire me and provide content in the form of archive material. My challenge was then to identify images and stories that would connect with the hearts of not only those from Liverpool but visiting tourists too. I am an incredibly indecisive person and at times can get overwhelmed when surrounded by a lot of choice; in hindsight, I spent far too much time going back and forth the same images, doubting my choices and wasting time. If I could go back and do it differently, I would have spent a lot less time in the Record Office choosing images and would have instead focused on learning about the stories behind certain images. In the end, I was drawn specifically to the more intimate photographs of people, as opposed to wide shots of the streets, these more intimate photos offered a very personal view into the lives of those living in Liverpool during the 1800s and 1900s.

However, I also had to think about how this content was going to be presented. Gazooky Studios and I had initially thought that an AR mobile experience would allow audiences to confidently explore the narrative with a familiar context; further, I enjoyed the juxtaposition that came with using modern technology to connect with the past. However, when pitching it to St George’s Hall they weren’t convinced. They were worried about older members of the public who might not have access to up to date smartphones or may be opposed to downloading an app or learning how to use one if it appeared difficult to navigate. Whilst I understood their concern, especially as a large portion of their current demographic is older, I tried to explain that it would be easy to make a very simple user interface. Furthermore, if the client was worried about access to the right technology, they could keep a small supply of handheld smart devices for members of the public who do not have their own. However, St George’s Hall did not like these solutions and asked for something different.

This was frustrating, especially as I am more comfortable producing AR applications than other immersive interactive applications, and I had spent most of my postgraduate exploring different AR experiences and learning to use Unity for the purpose of producing AR applications. However, it forced me to become a more rounded creative and developer as I adapted my skills and knowledge to suit the demands of the project. In the end, St George’s Hall, Gazooky and I settled on an interactive projection mapping experience that utilises elements of AR and spatial audio, taking inspiration from the teamLab Future Park, a venue filled with ‘play installations’ that encourage shared creativity and learning (Future Park, 2021). Such an exhibition still allows the audience to interact within a space and discover things for themselves whilst taking ownership of the narrative.

The idea was to have four interactive stations within St George’s Hall, with each station furthering the story of a young girl who is looking for her brother who had been arrested for stealing.

As most of the experience was now through interactive projections, it became important to not only consider how the photographs from the Record Office looked as they did on a computer screen but instead how they would look projected onto a brick wall. This would require testing, unfortunately, the projectors had not been installed this early in the process so I made sure to leave the Liverpool Record Office with a collection of different types of photographs - increasing my chances of curating the perfect images.

I also focused a lot of my research into finding news articles and personal extracts that linked back to St George’s Hall as the client had specified that they wanted St George’s Hall to be at the centre of the experience, supporting the audience’s journey through Liverpool’s history.

Capturing content

Once I felt comfortable enough that I had a well-rounded understanding of Liverpool’s history, I began assisting the scriptwriting and storyboarding process. This might have been the easiest stage of the entire process, with very few obstacles and a lot of creative freedom. Especially as it was decided that we would record the audio and the visual footage as separate entities. This allowed us to develop ambitious shots that were not limited by sound technology and vice versa - record really intimate, precise audio, without having to compromise on the visual performance. This then inspired us to develop a bespoke score that directly responded to the recorded audio.

The next stage in the project required strict organisation and time management as other people were giving me their time and equipment and I did not want to waste it. Therefore, I needed to know exactly what needed to be recorded and how it was going to be captured - taking into consideration location, availability, weather, and time of day. This was a complex part of the process and required a lot of attention to detail. There was a lot of back and forth as the schedules I was putting together kept having to change due to changes in the weather forecast and the availability of individuals. Working within a team can have just as many limitations as it has benefits - for example, it’s great to be able to share a workload, but it can become difficult if individuals become unreliable. Furthermore, more heads can mean more ideas, but it can also create chaos if people are on different wavelengths. Initially, I was overwhelmed by this task by all its moving parts, unsure if I would be able to execute it efficiently. However, I soon got into a productive rhythm, facilitating communication between different departments and finding a way to bring everyone together.

The shoot days themselves were fairly successful, we were only somewhat disrupted by the regulations around COVID-19. We did have to be clever about how we went about capturing the recorded media whilst staying two metres apart, or where social distancing could not be carried out, that we were carrying out any mitigating actions. Another factor we had to take into consideration was that one of the actors was very young so we had to ensure that she only worked one hour at a time; this meant I had to carefully prioritise how I structured the day and ensure that everyone was sticking to the schedule in order to stick to regulations and optimise productivity.

The Powerwall pt 1

After we had captured all the content, it was on to the editing and developing process. Three of the four spaces were self-explanatory and allowed us to take advantage of our environment. The audience would follow the young girl into three different prison cells as she looked for her brother, meeting the other prisoners as she does and learns their story and their experience. However, there was a long wall through the corridor in which the cells were kept that St George’s Hall had asked to be turned into something. We wanted this to be the part of the exhibition in which the audience could leave their handprint.

A very early concept I had for this part of the exhibition involved an interactive projection that mirrored the movements of the audience - perhaps in the form of ghostly shadows - supported by camera sensors. This would create the illusion that the audience is surrounded by the ghosts of those who once resided within the cells.

However, the sensors we had access to were predominantly touch-based as opposed to movement-based, meaning they respond much better to physical touch than abstract movement, making this concept difficult to execute.

Another idea involved creating ‘instagrammable’ moments throughout the wall, creating opportunities for play and encouraging audience members to take photos and share their experiences which would contribute to the marketing of the exhibition. For example, projecting different outfits onto the walls for audience members to step in front of and change their appearance - however, I was worried that this would take audience members away from the narrative and out of their immersion.

A third concept involved using the photos found from the national archives to allow audiences to create a soundscape around them. For example, if an audience member were to press on a picture of children running in the street, the speakers would start playing audio of children laughing - each photo would add something new to the soundscape until the audience was surrounded by the noise of a busy Victorian city, immersing the audience and transporting them through time.

Unfortunately, due to the fact that the projectors have poor speakers, that the prison cell corridor has very bad acoustics, and that noise will be bleeding out from the prison cells - this idea was very quickly eliminated

I found this process incredibly frustrating, I began doubting my capabilities and lost a lot of motivation due to the lack of success of my ideas. I started thinking to myself ‘how can I be creative if I am limited by the capabilities of the tech and the environment’. However, in hindsight, I realise that such limitations pushed me to create a more innovative and personal experience. Taking on all the notes from my previous pitches, I worked on developing an idea that would add to the story and world immersion, would not rely on any audio and would have a simple user interface involving touch-based sensors.

My final and successful concept involved a timeline of St George’s Hall’s history, told through stories and photographs to be submitted by Liverpool’s community. An idea like this allows us to celebrate our audience as well as involve them in the experience, whilst contributing to marketing and creating an accessible platform.

The client loved this idea and I was given full control over this part of the exhibition. I began by putting together a community call out - this involved reaching out to and interacting with members of the Liverpool community - gathering data and content whilst making sure I was complying with GDPR, curating and recording said data and content, quality checking the images and making sure I could use them.

The Powerwall pt 2

Once I had all the content I wanted to use, I designed a template for the timeline with Adobe Photoshop. Like all things to do with this project, it required a lot of back and forth and went through a few different versions, especially once I was able to start testing it on the projectors. This time, however, I made sure to trust the process and enjoy the opportunity to experiment.

The first version included a very busy design with lots of small images, but it soon became apparent that the smaller images do not look good on the projectors as they lose a lot of detail, especially the photographs of people as entire faces can be lost. This was very disappointing to me as I have previously stated, my favourite images were the more personal ones that gave us a glimpse into the everyday lives of individuals. Therefore, it was important that I created a template that allowed for fewer and larger photos.

For another template, I used the backdrop of a blue sky and instead of having lots of small images on the wall, I had scattered letters that looked like they were blowing in the wind. The idea would be that when an audience member selects one of the letters, an image would appear, taking up the whole section of the wall, allowing for the audience to appreciate nuance detail. This got rid of a lot of clutter and meant I didn’t have to compromise on the image as I no longer had to worry about how it looked when small as audience members would only see it once it expands. However, the client was worried about the user interface; they were unsure if the older participants would know to press on the black letters like a button. I suggested including a programme that told the audience how to interact with each part of the exhibition but St George’s Hall was very clear that they wanted the entire experience to be digital, they didn’t want to have to rely on any physical artefacts. I also suggested that there is a possibility that the audience will simply be able to pick up the call to action and know what to do without explanation, especially with the understanding that this is an interactive exhibition.

Nevertheless, the client did not want to take any chances and asked for something a little more enticing. I was disappointed as this was my favourite design, but I understand that a big part of a creative process like this is being able to compromise in the face of client instructions, it’s not just about what I want. Therefore, I went on to design a template in the form of a newspaper that evolves with the timeline, within each era are four images that when pressed, expand to take up the space of the whole wall. Even though the image has the option to be big, I still wanted images that would look good when small, so I had to carefully curate images that told a story, celebrated Liverpool's community and looked good when both small and large. This became quite time consuming as I kept having to update the Unity build and re-upload it, but I understand that trial and error is necessary when researching and developing.

Whilst I was working on the template of the powerwall, I was also liaising with the technical director of Gazooky to build a Unity application that would allow audience members to interact with the images. I wanted something that would be very simple yet effective; buttons are one of the most commonly used UI components, they are easy to customize, quick to configure and simple to navigate. However, what became a big obstacle was incorporating the third party software needed to make the application work on a projection triggered by touch-based sensors. If I had it my way I would have used Isadora - Isadora is an incredibly user friendly and flexible programming environment with an accessible customer service platform, whereas the third-party software we ended up using created a lot of issues along the process. Due to its bespoke nature, only the company who had built it understood it - therefore, whenever we had any problems such as the touch sensitivity not working, we had to wait until someone from the company could come and look at the problem and troubleshoot it. This hugely disrupted the process and meant that there were sometimes days where I was unable to carry out any work until the problem had been fixed.

conclusion

Despite this, I am really happy with the final result, I feel it achieves what the client wanted whilst staying true to the narrative. With a simple user interface incorporating innovative tech, references to St George’s Hall and a moving and accessible narrative. It’s been a really interesting learning experience working so closely for a client; I’ve learnt there’s a really fine balance between creative freedom and fulfilling a brief. I was able to take a lot of what I learnt from the collaborative project, but this time around the client was a lot more hands-on. It took some time for me to adjust but I eventually found comfort within the structure of what was being asked of me. Furthermore, I benefited a lot from having a party of relevant professionals who would frequently offer feedback. At times, their feedback was overwhelming or sometimes confusing, but I made sure to allow myself the space to digest what was being asked of me.

Using project management software I was able to break down and categorise tasks as well as prioritise my workload based on deadlines and level of importance. This project always had a handful of moving parts, meaning it was imperative that I stayed on top of my organisation and time management. There have been times during this process where I have fallen behind my organisation and it has had a direct effect on other members of the team. For example, I was recently sent an email with a new Unity build to upload at St George’s Hall - however, I did not see this email for a few days which meant it didn’t get uploaded until after a crucial test run, causing confusion and frustration. In order to prevent a similar mishap, I have set an alert on my outlook which will send a notification to my phone and smartwatch when I receive emails from particular contacts. I have also set up a folder where those emails will automatically be sent to, and each evening I will check that folder and write up any tasks that I have been given.

Over the next month, I intend to work on making the exhibition more accessible by adding subtitles brief summaries for each part of the exhibition that could be made into brail or audio descriptions. I also plan to carry out audience testing; in an ideal scenario I will be able to involve a diverse range of demographics, but I am most interested to see how older participants will engage with the experience, to see if they will know when and how to interact with the different projections. I’m really excited to see how an audience will engage with this experience as it has been designed in a way that celebrates their involvement - I want them to come out of the experience feeling as though they were at the centre of the narrative.

During the module, I have developed skills not only as a creative and developer but also in terms of working in a professional environment under strict deadlines and budgetary restrictions. The biggest thing I have noticed that I struggled with is that I am easily overwhelmed, which causes me to be unproductive. I half expected this project to be a steep learning curve, but I can not let myself get lost behind new challenges. The first few times I felt overwhelmed I tried to deal with it myself - but after I noticed that it kept happening I reached out to Gazooky to explain where my head was at. I was nervous to do so, but I was awarded support and flexibility. I was told that it is okay to share responsibilities and to say no to favours, it was also suggested to me that I break down bigger tasks into smaller steps and approach them one by one as opposed to trying to tackle it all at once. From this experience, I have become better at managing expectations and will take what I have learnt with Gazooky onto other projects. In the future, if a manager asks for things out of my remit I will take the time to digest and potentially delegate what is being asked of me.

Bakare, L. (2019) UK's Regional Theatre At Risk Amid Funding Cuts, Say Directors [Online]. The Guardian. Available at: <https://www.theguardian.com/stage/2019/dec/02/uk-regional-theatre-at-risk-amid-funding-cuts-say-directors> [Accessed 26 April 2020].

Culture Track (2017) Culture Track '17, New York: LaPlaca Cohen.

DCMS. (2021). DCMS Economic Estimates 2019 (provisional): Gross Value Added. [Online] GOV..Uk. Available at: <https://www.gov.uk/government/statistics/dcms-economic-estimates-2019-gross-value-added/dcms-economic-estimates-2019-provisional-gross-value-added> [Accessed 26 August]

Digital Signage Today (2016). 'Making the Customer Experience Interactive', Digital Signage Today [online]. <https://www.digitalsignagetoday.com/blogs/making-the-customer-experience-interactive/?printable=true> [Accessed 30th October 2020].

Future Park. (2021) About Future Park. [online] teamLab Future Park. Available at: <https://futurepark.teamlab.art/en/about/> [Accessed 03 September 2021]

Gecko. (2014) About Gecko. [online] Gecko Theatre. Available at: <https://www.geckotheatre.com/about-gecko> [Accessed 13th February 2021]

Kirkham, N. (2019) The Benefits of Attending Live Performance for Children and Adolescents. Encore Tickets.

MTM. (2018) Live-to-Digital in the Arts: Across music, dance, opera, visual arts, literature, combined arts and museums. University of Exeter

Ofcom. (2020) Media Nations: UK 2020. London: Ofcom

Richardson, D. (2019) The Benefits of Attending Live Performance for the Aging Population. Encore Tickets.

Snow, G. (2016) Audience Agency Data Shows Ageing Demographic Of Theatregoers [Online] The Stage. Available at: <https://www.thestage.co.uk/news/audience-agency-data-shows-ageing-demographic-of-theatregoers> [Accessed 29 April 2020].

TRG. (2019) Sales Data Headlines 2018. UK Theatre.