For our capstone project, we are attempting to create an immersive, relaxing space in the ATLAS building on the CU Boulder campus. The space will have nature-inspired elements, such as soundscapes and lighting, as well as an interactive fractal visualization that the user can manipulate in real time. The primary goal of the project is to create an immersive, calming space free of overstimulation, where people can relax regardless of whether they are actively meditating, interacting with the fractal visualization, or just enjoying the quiet sounds of nature.
As a first step towards making this project a reality, we embarked on a prototyping phase to test out different components of our idea. Goals for this phase were to determine answers to questions we had about specific technologies to use, how certain elements would look and feel, and how the overall experience would play out for a user. Many of our prototypes were designed to serve as proofs of concept for technical components, while others were intended to help us more intentionally design the experience overall and clarify our vision for the project.
These should help answer specific questions about your idea, for instance:
- Does the technology exist to even make this happen? How accessible is it for you to work with?
- Can I learn / utilize the tools necessary in the given time?
- What could the interface look like?
- What does it look like to walk through the interaction with the project?
Be sure to document the work in a multimedia format (include sketches, renderings, screenshots, videos, etc.).
Technology for User Controls (Implementation)
A large part of our project will involve controls that the user manipulates to achieve things like setting up the ambience of the room (lighting and soundscapes) and changing aspects of the fractal visualization. Therefore, our first prototype was intended to explore various technologies to determine which types of controls might work best and be the easiest to implement. This prototype was primarily an implementation prototype, as it focuses on how the controls will actually be programmed and developed.
Phone Slot (Switch)
In order to get anything interesting to happen inside the room, the first thing the user will have to do is place their phone into a slot. This action acts as a “power switch” for the rest of the installation, as the user should be disconnected from their phone in order to get the best results from the experience.
To test how this phone slot would work, we first constructed a rough cardboard prototype of the slot itself, and then inserted some trigger switches inside of the slot to see which configurations would produce the most reliable results.
Using our Arduino MEGA microcontroller and a ring of Neopixels, we tested putting the trigger switches in the bottom of the phone slot and on the side. Below, you can see the test of putting the switch in the bottom of the slot. When the phone triggers the switch, the Neopixels light up green.
We also wanted to try putting the switches on the backside of the slot, to see if it would function any differently. The result can be seen in the video below.
Although both switch configurations worked okay, we determined that the bottom switch position was better for a few reasons. First of all, having one switch in the bottom center of the slot will allow the phone slot to work more reliably with a variety of different phone sizes. Secondly, the bottom position looks much neater, as the wires are hidden away without having to build a second wall to hide the back of the switch itself.
Ambience Controls (RFID Reader & Tag)
Next, we wanted to explore RFID technology as a solution to the ambience controls of the room. Our original design includes having various squares on the table, each with a different icon of a nature environment. The user can place an acrylic cube onto the different squares, and the lighting and sounds of the room will change to fit the environment of the selected square. For example, if the user decides to put the cube on a square with a thunderstorm icon, the lighting will become darker and the soundscapes will be comprised of rain and soft thunder.
To create this effect, we knew we would have to utilize some technology that could easily identify different pre-designed inputs. Since we wanted to include the tangible interaction of physically moving a cube around, we decided to look into RFID technology for this aspect of the project. However, neither of us had ever used RFID before. We wanted to create a prototype that would ensure the technology would fit the desired application and help us prove to ourselves that we could utilize a new technology without too much trouble.
To start this prototype, we ordered a cheap RFID kit from Amazon that included an RC522 reader and five RFID tags. We also constructed another cardboard prototype that could simulate the action of moving a cube around to different “environment” squares, which can be seen below.
Once the RFID kit arrived, we soldered on some header pins and followed a tutorial to start hooking up the RFID module to our Arduino MEGA board. We included a handy RC522 Arduino library in our code to get started. At first, the tutorial didn’t work, and we realized it was because the Arduino MEGA uses different pins than the Arduino Uno, which was being used in the tutorial. After reconfiguring the pins, we were able to load up an example from the RC522 library (using this tutorial) that printed out tag information to the serial monitor when a tag was placed on the reader. Finally, we got the example to work!
We wanted to take this prototype one step farther by getting our Arduino to change the color of an LED strip based on which tag was placed on the reader. This is similar to what the final application of the RFID technology will be: changing the lights and sounds of the room based on which “environment” is selected. Many of the tutorials we tried were pretty complicated, but we eventually found this one that keeps things simple by just printing out the id of each tag and nothing else. From there, we were able to figure out what the ids of our different tags were, and write a simple program that changes the Neopixels based on which tag has been scanned.
Our code for the test program can be found below:
Note: Although our original idea (and prototype) was to have multiple squares and one cube to be moved around, we have since reversed the idea to include multiple cubes (each representing a different environment and with a unique RFID tag) and a single square (with an RFID reader) to place the cubes onto. This makes more sense for the RFID technology, because we can have one RFID reader hooked up to the Arduino beneath the table and out of sight. Meanwhile, the cubes can be moved independently and do not require being hooked up to power.
As a result of creating this prototype and sample code, we have finished a proof of concept for using RFID technology in our project. We got started using the RC522 library in ways similar to our final application, and decided to change our original idea to include multiple cubes with one main square on the table. Moving forward, it shouldn’t be too difficult to implement the next steps: getting lighting patterns and sounds to change upon scanning a new RFID tag.
Flow of User Experience (Role)
For our next prototype, we wanted to focus on the bigger picture and develop a vision for what a user’s experience would be as they interact with out installation. Essentially, the intended interaction will be as follows:
- User finds out about the “Flow Room” exhibit somehow, whether through advertising, word of mouth, or something else.
- User finds the room in the ATLAS building where our exhibit is installed, and enters the room.
- Upon entering the room is lit dimly, with the only real attraction being a table at the end of the room. On the table, there are a few different elements, but the only one that is lit up will be the phone slot (see sketch below). This is intentional – in order for anything else to happen, the user needs to put their phone into the slot.
- Once their phone is in the slot, the phone slot lights will turn green and the next element will light up (the “environment” square for setting the room ambience). Once the user puts a cube onto the square, the lighting and sounds of the room will change accordingly, the square’s lights will turn green, and the next element will light up.
- The third and final element to light up is going to be the main controls of the game. Once the user touches the “Start” button, the controls will light up green and a projector will slowly turn on, projecting the fractal visualization on the wall in front of the user.
- At this point, the user can use the controls to manipulate various aspects of the fractal visualization. There is some evidence that looking at fractals can have a calming effect on people. This, combined with the nature-inspired ambience and disconnection from overstimulating technology (like phones), will hopefully calm and relax the user for as long as they would like to stay in the room.
- At any point, the user can stop the fractal visualization and just enjoy the nature sounds, or even remove the cube from the “environment” square and just be in a quiet space. We want to provide users with an experience that fits their needs, whether that experience is more passive (being in a quiet space, meditating with nature sounds, just relaxing) or more interactive (manipulating the calming fractal visualizations).
- Once the user takes their phone out of the slot, the installation “powers down” and returns to the state it was in when the user entered the room.
A storyboard of how the user might interact with our installation can be found below.
In addition to the “role prototype” of the user experience storyboard, we wanted to get a clearer picture of all the technology that might be used to create our installation, and how it all fits together. We started with a rough sketch of all the different components, and then refined this slightly to create a “blueprint” of the technology needed to create this experience.
By creating these prototypes for the user experience and the technology behind the installation, we were able to clarify our priorities for the experience and lay out a clear plan of what is happening behind the scenes to create that experience.
Interactive Fractal Visualization (Implementation)
In this portion of our prototyping phase, we focused on learning how to create nature based spiral, fractals, recursive patterns, animated sketches, and practiced serial communication in p5.js. Overall our goal here was to take steps in developing our p5 experience and get a better idea of what the interaction would be (user input methods & what the user changes). We created many small p5 sketches to gain understandings of all of these concepts, and now we can begin to combine them for our project’s interactive animation.
Phyllotaxis
First, we followed a Coding Train tutorial to make a phyllotaxis spiral that grows to generate a pattern replicating spirals found in many plants. The placement of new circles and the coloring of the circles are dependent on the angles at which they are located from the center which builds the spiral-shape. This was a good first step because it was a simple introduction to how angles exist in nature, how shapes are built off of those angles, and how to represent this visually in p5.js. When the angle (variable “a”) is changed to different angles existing in plants, the spiral changes. Here is what the spiral looks like when the sketch runs for a little while:
Here is the code for the simple phyllotaxis sketch.
Golden ratio spinning circles
Next, we looked into implementing the golden ratio into a moving p5 sketch. The golden ratio exists everywhere in nature, and it creates very nice visuals. In this sketch we are drawing new circles on the outside of the design using the golden ratio, and rotating the existing circles incrementally. This was a helpful exercise in smooth frame transitions, as well as more practice in getting comfortable using angles and equations in p5. Size, color, and distance between the circles can all be modified by changing initial variables. Here is a screenshot of what the sketch looks like after running for a while.
Here is the link to see the sketch in full motion!
Serial input with spinning circles
Next, we built on the spinning circles animation and incorporated some serial input from a potentiometer. This was an important step because it got us thinking more about what we want the user interaction to be (for instance, what controllers we want them to be using) and what those inputs will be changing in the code. But for now, it was a good practice to reinforce our serial communication skills, and to begin to see where the physical interactions could intersect with the math of our future fractal animation.
Here is the link to the p5 sketch with serial communication set up! The Arduino code for this section was very simple, all it did was send the potentiometer value to the serial port.
Recursive circles
Next, in the spirit of fully understanding how to generate our own fractals, we looked into creating recursive patterns. To start, we followed this Coding Train tutorial to learn a simple strategy to generating recursive patterns with p5.js. The math here is pretty simple, but it is important to get practice manipulating shapes in p5 (and how they are drawn) using the math. However, we don’t think this is the best strategy for drawing recursion. It takes a lot for the computer to draw the pattern, and when we tried implementing sliders to make it responsive it was not smooth at all. Although it was fun to explore how build the patterns! Here is a screenshot of what the recursive circle code builds:
More Recursion
Next, we decided to look into different ways to use recursion in p5.js. We used a sketch found on openprocessing.org as a guide, and played around to see what each of the variables were doing for the visuals. The code we looked at in the processing example was very complex, so we just tried to trace the program to find where math we recognized was being used. After playing around with this code, and the recursive circles sketch above, we realize that generating designs like this take a lot to render, so maybe serial communication isn’t the best way to manipulate this design. However, adding motions to sketches similar to this with slow, smooth frame-rate transitions would be just the sort of relaxing visuals we are looking for!
Here is the link to that recursion sketch.
Nautilus shell . spiral
In this example, a series of curves are drawn recursively using p5.js’s curve() function. First, the top and bottom curves are drawn that will encapsulate the whole shell. Then, the functions to draw the curves inside are called recursively to build detail in the shell.
Here is the link to the shell sketch. We also learned that there are other ways to draw curves and arcs in Processing. The curve() function uses control points to create shape of the curve, which can be kind of overwhelming to try and finesse to get it right. So, this leads us to the next small p5 sketch we wrote, involving Fibonacci squares.
Fibonacci squares (variables!)
Here we built another representation of a spiral, but with squares. While this sketch visually shows the golden ratio, it was not used in the math. Another important part of this sketch was our use of variables rather than hard-coded numbers. This allowed the squares to build on themselves, and we potentially could have made the spiral as big as we wanted (or variable with serial input!)
An easy way to incorporate curves into this design would be to use p5’s arc() function. arc() takes 6 parameters, and it is easy to fit a quarter circle into a square across the diagonal, which would build a Fibonacci spiral representation.
Here is the link to the Fibonacci squares sketch.
After p5 prototypes…
It was really helpful to make these 7 mini p5.js implementation prototypes, because now we can be much more informed when making design decisions for what we want the end visual product to be. We learned a lot about the intersection of recursive functions, math, and shapes in p5, and are excited to explore more deeply how our interaction design can intersect with all of that.
Neopixel Colors & Transitions (Look & Feel)
For our last prototype, we wanted to work with Neopixels, which will make up many different components in our installation. This prototype focuses more on the look and feel of what the user will experience. We worked on creating colors for the various ambience “environments” and experimented with fabric to see how the Neopixel light would be diffused.
The first step was getting the Neopixels hooked up to our Arduino and experimenting with material to see how the light would be diffused at different distances.
Next, we wanted to explore different colors that could be used for the different environments. Since the environments are nature-inspired, we used greens and blues for the settings with more water and foliage, and used warmer golds and tans for settings like the desert. You can see some potential color options below:
While programming the colors was pretty easy, most of the time and effort for this prototype was spent trying to develop smooth transitions between colors, as the user changes the environment. This proved to be much more difficult than expected. We ran into quite a few limitations of the Adafruit Neopixel library, and tried working with other plugins like Neopatterns to help make transitions smoother.
To be honest, one of the biggest takeaways from this prototype is that we might have to look to other libraries to achieve the fading and transition effects we want to create. We spent way too much time trying different ways of creating fades programmatically, with little success. Finally, we figured out a way to “pulse” the Neopixels for as long as a switch is active, but it isn’t the smoothest.
Part of the goal of prototyping is to figure out which technologies are feasible and which ones aren’t, and this prototype is a good example of when not everything works out. We definitely learned a lot throughout the process about mapping functions with pixels and even different colors and diffusing effects. However, after much research and experimentation, we will probably switch to a library like FastLED moving forward, because it offers more advanced functionality for what we want to accomplish.
Below, you can see the code we developed for the pulsing Neopixel:
Nate and Elsa's Capstone 