When AI meets XR — Delta Reality Hackathon 2023
About a month ago, we held an exciting hackathon in our office, where three teams battled it out to create a solution that merges XR and AI. The hackathon was aptly named “XR Meets AI,” and it was truly a blast to see all the creativity flowing as the teams dove into the challenge. The goal was to find new and innovative ways to leverage the potential of AI in our industry, and the results were impressive. In this blog post, we’ll take a closer look at the hackathon, the teams, and their innovative ideas.
AI overture
Snacking and building up the strategy was a great start to the day, and judging by the energy of the participants this early in the morning, this hackathon had all the predispositions to kick it off right. The three teams, Meteorologists, Seismologists, and AI Anarchists, were already formed beforehand.
The competition started with presenting the rules of this hackathon, followed by the jury’s expectations of the solutions each team had to present. In short, the event was 24 hours long, and in that period of time, the team was supposed to complete their ideas and turn them into feasible solutions that the jury, and the rest of the team, could try out on their own. The challenge was daunting, but we had confidence in our team’s abilities. The main idea, as we mentioned before, was to utilize AI to produce either a product, a solution, or an app. Among the wide variety of currently available AI software, we were extremely excited to see what the teams would come up with.
Creativity at its best
We kicked it off at 9 AM on Saturday, and the plan was to have a complete solution done by Sunday at 9 AM. While AI Aranrchists took their planning to the nearby cafe, the remaining two teams worked on their strategy in the office. It was interesting to watch their approaches as the three teams consisted of team members from the same department. Soon after the plans were set in stone, all three teams dove into working. First, each team outlined the problem they wanted to solve and made a rough blueprint for their app. Finally, they set the priorities of the end goal they wanted to achieve. It was incredibly challenging for everyone, especially for the Seismologists team, who were only two developers. Team AI Anarchists also had a demanding job since they consisted of four artists, but AI was there for any of their development problems. Fifteen hours later, apps started to look like something. Problems appeared for all three teams, but overall, AI came in handy, especially in the early morning hours when their focus slowly drifted away. Food and beverages were available for them at all times to help them get back on their feet. Still, it was 3 AM when some decided to doze off. The bravest stood up all night and worked on their pitch as the deadline slowly started to creep in. 24 hours later, these were the applications they created.
Meteorologists’ Muse{AI}um
The team that consisted of three Unity 3D engineers — Mario, Sebastijan and Bruno — created a unique experience that enables you to visit a museum in VR and watch an exhibition that you want. For instance, if you want to check out an art exhibition inspired by poisonous plants, this is exactly what you would get. You can choose any subject you want. It all works in a way that you talk to the virtual curator, and your input is processed by speech-to-text, connected to ChatGPT to get an answer, and then spoken to the user by text-to-speech through the same curator. All of that was perfectly rounded by the actual art exhibits in the virtual room, which was made using speech-to-text, followed by ChatGPT to make the image title and the description, and finally processed by DallE to get an actual image. The image titles and descriptions were processed by DallE to generate actual images, which were then sent to ChatGPT to provide explanations for each art piece.
AI Anarchists’ Journey Through the Art of Vincent Van Gogh
A team of four artists created a VR experience in which the user can interactively explore Vincent Van Gogh’s artistic expression and his state of mind. Ksenija, Luka, Petar, and Ivo chose him as a post-impressionist artist with a unique art style and artistic language.
This team lacked a developer to help them out with the code. However, by relying on AI, that segment didn’t lack at all. The experience starts in a VR museum where you meet Van Gogh’s portrait, and by simply interacting with it, you get to enter a whole new world. Through the environments presented to you, you can explore his artistic style, but also his emotional and physical states, heavily enhanced by audio-visual effects. The goal was to bring this artist closer to the public, but by delving deep into his mind, this app also creates a feeling of empathy.
The app was created using five AI tools — ChatGPT for scripts and content, Midjourney for paintings and concepts, Stable Diffusion for all the textures, Text-to-Speech and Speech-to-Text to enable dialogs and conversation, and finally, Beatoven for the music and sounds.
Seismologists’ PersonalHorror
Fears are one thing that constantly surrounds us. Every single person has one, and that is why Josip and Josip decided to create an app that helps you overcome your fears step by step. These two developers put together an application that solves “hard-coded” assets inside the app and enables asset personalization in a way that you select the fears you have at the very start. The experience starts in a room filled with mirrors and other elements onto which images are projected. Depending on the fears you choose, you get different images generated by using Stability AI. For instance, if you choose to have a fear of heights, you are placed on a platform with a horizon surrounding you. It is much easier to face such fear when you know this is all happening in VR, and this kind of experience eases you in so that you are ready to face it in reality. The experience has an intro and outro, for which they manually used ChatGPT.
This is one of those apps that could really help people around the world, especially in the times we live in, times when fear creeps in the most unexpected ways.
And the Oscar goes to…
After watching their pitches, our judges — Vedran, Darian, and Jake — were extremely pleased with the ideas and the outcomes. Picking a winner certainly wasn’t easy, but Meteorologists eventually took first place. Ultimately, both Seismologists and AI Anarchists’ solutions were so amazing that they shared second place.
This was our first hackathon so far, but after such a positive experience and three amazing ideas brought together in just 24 hours, this will be a yearly event. We might even open it to the public in the future. Who knows. The one thing that matters is that AI is omnipresent and will continue to be. There is a solution for everything, but it still requires skill to use it properly. Today’s choice is not really a choice, as you need to adapt and harness AI’s benefits to avoid being left behind.
