AUGMENTED REALITY

WEEK 1

LAUNCH

We launch the Augmented Reality project in the morning and we immediately started to set up Vuforia on Unity. Since the process is quite complicated at first it was nice to have a depth explanation of all the steps of the setup. As soon as everything was working we started to try out Vuforia with our computer webcam and then learned how to transfer it on a tablet provided.

The launch was interesting as we saw a lot of references and what kind of stuff we could create in this project. To be honest I wasn't expecting it to be so powerful. Unity and AR are very powerful and I really underestimated that.
We also reviewed the submission details and some requirements of the project such as the Android only policy as setting up a Unity app on an iPhone can be quite tricky.

WORKSHOP

This project was 3 weeks long, I really liked that as I had the opportunity to think about my concept more and experiment more stuff. The first week was full of workshops with Leon and studio support.
In the workshops ran by Leon we learned even more about unity and Vuforia. We discovered a whole new set of possibilities of Unity and Vuforia like having an API and use data to animate some object. Use sound as a trigger etc.

These workshops were really helpful to really understand the possible path we could take for this project.

IDEA & RESEARCH

Towards the end of the first week, I started to think about what I could do for this project. I had an idea in mind for quite a long time, to do an animated poster. So basically to bring to life a static poster.
Also, since the majority of my last submissions were quite focused on a physical artefact (control, design domain, and partially web apis) I really wanted with this project to go back to graphic design and/or 3D.

Sketchbook notes and drawings

I had 3 ideas over the first weekend.

POSTER
The first one was, as I said, to have a poster based work. But instead of simply bringing to life an A3 poster (used as an image target) and then animated the same one and display it in AR, I wanted to use space as a canvas. So basically using walls, tables, doors, windows, etc as a canvas for my posters. I wanted to explore some kind of unconventional sizes of canvas and create animated poster with these sizes. I quite liked the potential outcome of this as it involved a lot of graphic design and at the same a very experimental way of using it with "weird" canvas sizes.

FLOWERS
The second idea was quite simple. I wanted to create 3D flowers in Maya or C4D and then display them in AR. Though I wanted to have an almost psychedelic vibe to it and change the hue completely. Instead of having normal flower colors I would add very unusual and combinations of colors. Also, I wanted to display them with a colorful background instead of using a real-world background. Though I think this idea was maybe too literal and not researched enough. The concept behind wasn't really enough for me.. and I also didn't know how to create really good flower models in 3D, something that I should focus probably over the summer break. I think it needs more reflection

AUGMENTED TEXTURES
Then the third idea was simply to augment the reality with new textures thanks to AR. So basically instead of seeing a wood table, for example, I wanted to replace the wood by another texture. Probably with something I would tweak in Maya. I liked the idea but didn't know as well really how to proceed with this. Maybe something to look forward to next year?

REFERENCES

Here are some references that lead me to the different ideas I detailed down above.

Use of space


Graphic design & colors




Flowers


Texture augmented


WEEK

TUTORIAL

In the second week on Monday, we had a group tutorial to expose our current ideas and reflect in a group on them in order to develop them. It was also a good meeting point to ask some questions about some technical stuff.
So I talked through my three ideas and we discussed a bit of each. I learned that I could use the Maya paint tool to actually create quite detailed flowers which would be very useful. I also asked about the potential ways of triggering animation in unity, for example, if a tap on the device could trigger something.
I was quite excited to hear that we could! A lot of potential animations were possible.

I realized it would still quite complicated to go on with the flower idea as the model is that accurate if we zoom in. The paint tool is mostly used to decorate big landscape. Not for macro purposes especially with small objects like flowers

EXPERIMENT

After the tutorial, I decided to stick with my poster idea, as I really liked the concept of unconventional canvas and sizes. I started to take measures of potential areas in the studio that I would use for the project like walls, tables, doors, and some plinths.

some measurements of the studio's walls, table & plinths

I took the remaining days of the week to assemble a collection of graphic references and started to design some quick poster and animation to be able to test it quite fast. I think it was important to have something to test for the upcoming workshop with Leon, so we could test things and go forward with the project while he was here to help.

I started to create some posters and experimented with things in Photoshop. I really liked this book cover and wanted to try this in a poster. I thought it would be interesting to have some carved texture on a poster that is digitally projected on a wall. I would feel like the wall is carved.

book cover reference

photoshop setting I used to recreate this carved effect

On Friday, I attended Leon's morning workshop. It helped us to view our AR projection with a tablet instead of our computer's webcam. I then wanted to put my poster animation on the tablet. This is where I sadly realized that it wasn't working. It only showed a white plane instead of the poster animation. To sim up the situation, I spent the whole afternoon trying to fix it with scripts etc and the help of Leon, but nothing could help it... I restarted new projects from scratch followed tutorials on youtube etc.

I used this as a target

and this poster I made this second week.

I decided to have a look over the weekend with this time different versions of Unity.
I continued to struggle the whole day on Saturday with the tablet without any successful results... I tried with 3 different unity version 2019/2018/2017 and nothing worked for me weirdly. I followed step by step different tutorials and it didn't work...
So I had to find a solution. I could either change my concept and display only image or 3D objects or I could potentially try to build it on IOS.

I remembered some lectures I had in France about Xcode, it was complex but I think I could quite easily work with it for this simple build.
I followed a tutorial on youtube on how to build the unity app on IOS. Obviously, it didn't work. It crashed on launch for several times, I had to try different scripts, player settings, IOS versions and different versions of unity. It finally worked with a 2017 version of unity and a particular version of Vuforia.

Error logs of my first attempts with the IOS build.

Signing up step you have to go through to run the app on your device.

Then sadly the COVID19 got really serious this second week, so I quickly brought back the tablet on Sunday and grab some personal stuff from my drawer in case the school was closing.

WEEK 3

COVID

We learned that all upcoming submissions were canceled. I was really sad because I finally found a way around to make my concept, but safety and health first.

So I continued to develop the project until the summative assessment. I was initially planning to use the studio space as a canvas but I to revise that and take my home in France as a canvas. Since this situation brought me a lot of stress as I had to plan my way home last-minute before France closed its border and prepared to get into quarantine.
I also wanted to revise some other project and LJ posts for the final deadline the 26th so I couldn't finish properly my project. I still made a functioning app with an animated poster, sadly the sizes were adapted to some of the studio spaces and not my home. Though the total lack of studio support really makes it harder to go forward relatively fast, as I had to troubleshoot myself and sometimes without the right method. Though I think I managed to do all the big requirements of the project almost alone. I was quite happy to troubleshoot everything myself, of course, I had more time to fix everything up.

FINAL CONCEPT

The final concept was to have a series of target image with the letter A Y X I on some orange background. AYXI is an abstract name for the project. Each letter would trigger a poster that is adapted to space in the studio. For example, an entire wall side would be replaced by a big animated poster. The poster themself are very abstract though, there's no continuity or theme. Just some random inspiration from my current mood.

I used the letters of my project name as target images.




POSTERS

animated posters


static version


The 3 posters in the Unity editor. (4th smallest square was a blank test)

'ALMOST' FINAL WORK

This is a recording of the IOS build featuring 3 animated posters layed around my flat kitchen (instead of the studio). The posters don't fit the surface because I didn't have the time to recreate each poster for my kitchen space.

As I didn't have the studio space I thought it would be unnecessary to document it and edit a video without the right space as the concept would not be shown properly.

REFLECTION

I think this project was a great opportunity to look at this theme of augmenting our reality with digital things such as digital pictures and 3D objects. It questions a lot about the use of technology to visualize objects and content in our environment. I often asked myself if this would be the kind of future we'll have to look into for ads or entertainment.
I found very interesting to have an unconventional approach for graphic design and create posters with new rules. I think it's a great way to expand a practice and also learn new things with these challenges.

OVERALL THOUGHTS

It was a great almost full project. I really enjoyed learning this new piece of technology and would love to experiment more with it in future projects. It had a huge potential that I haven't acknowledge until now. Quite sad to see it end like this because of the covid19, but I'm sure I'll find another way to do a full project with AR!