Personal Project 1
8 April, 2019 - 12 July, 2019 (Week 2 - Week 15)
Jesslyn Fabryando (0332213)
Personal Project 1
RMIT Collaboration
8 April, 2019 (Week 2)
On the first day of collaboration, we get to know each other with the RMIT students and moving on into the briefing session by Dr. Li Ping. The theme that were given to us is about 'Cities'. Afterwards, we were asked to write down our names on what we are specialized in. At first, I am decisive of specializing either Audio or Interactive but ended up choosing interactive as I have experience doing coding in other modules. From there, we chose our group members and formed a group called 'Mirrorless'.
Group member’s specializations
- Jesslyn - Interactive
- Jaslyn - Animation
- Joel - Video
- Loui - Audio
- Umar - 3D modelling
 |
| Fig.1. Mirrorless group picture |
On that day itself, we started to think about our concept ideation. We started to brainstorm on possible projects for the given brief, themed “Cities”. We opted to go for a more artistic approach for the project.
Upon brainstorming we came up with a few ideas:
- Showcase of the building of a civilization.
- The comparison of two cities, Melbourne and Kuala Lumpur.
- An aging city
- Cave turning into a city
Eventually we came up with the idea of expectations vs reality. From here we decided to show the contrast of lives and conditions in the city, the good and bad. The idea is to show that in everyday lives all we see as a society is only the best out of a particular situation, whereas in reality behind the curtain is a side of the city that is not as fortunate.
 |
| Fig. 1.1. Brainstorming ideas |
The execution for this will be by using projection mapping in an enclosed area, done using white sheets projected from the outside to prevent shadows obstruction, with one side showing the good part of a city while the other shows the bad part. Throughout the whole show, users would only be able to see the good part no matter which screen they look at, while the other screen behind them would be showing a different visual. The idea is that the users would not be able to see what is going on behind them unless they use a mirror.
 |
| Fig. 1.2. First Presentation on rough ideas |
A small presentation was conducted pitching our rough ideas. Everyone is participating in giving some feedbacks especially the lecturers. The lecturers were pretty satisfied with our idea and concept, however after pitching the idea, our main concern at the moment is how do we keep the concept going without the use of handheld mirrors. That is a problem that we need to solve.
9 April, 2019 (Week 2)
On the second day of collaboration, we decided to brainstorm again to make the topic and the content even more solid. For now, our idea is there but there are certain problems that we need to solve. We decided to finalized our idea and concept by today in order to be able to present it on the next day to finalized it and start doing the prototype.
Also, on that day itself we went off to Pasar Seni to get some inspirations by looking around on that area.
 |
| Fig. 2. Pasar Seni trip to get some inspiration |
As we got back to campus from Pasar Seni, there was a short talk about VR from Studio Behind 90 where they also demonstrate how to illustrate in photoshop. Afterwards, we continued on our discussion and progression. Upon consultation with the lectures since we still haven’t figured out by keeping the concepts without the use of handheld, they suggested to change execution by using VR instead of projection mapping. For the upscaling one, the projection concept can still be used as the first person is experiencing the VR the audiences can see what’s inside the VR. The VR concept is to solve the problem of detecting the sensors of when the person is turning their back to see a video of the bad side of the place.
 |
| Fig. 2.1. Small presentation from Studio Behind 90 |
 |
| Fig. 2.2. Discussion |
 |
| Fig. 2.3. Trying out Oculus |
We were reminded to come up with a group name as well as the project name. Since, our idea is having a problem to get rid the use of mirror we came up with the group name ‘Mirrorless’ as we found the solution through the use of VR. As for the project name, we are emphasizing with the problem around us which is ‘ Ignorance’. The problem that we raised is about people that is their human nature tend to ignore small and insignificant things around them. Specifically, we focus on the problem of environmental and social issues that people tend to ignore. From this problem, we came up with the project name ‘ Buta Hati’. The word Buta Hati is a proverb in Malay which means ignorance. When it is directly translated in English it is called ‘Blind Heart’.
10 April, 2019 (Week 2)
On the third day of collaboration, early in the morning we still haven’t had a solid idea and we haven’t finalized what outcome we are going to produce because when we were suggested to do VR we still want to ask how does VR works. We need to experience how VR works and that’s the time we know and figure out to make one. Another thing is that, we still keep on questioning what do we mean by good and bad things that we want to show.
One of my group member, Umar, did some mock-up of how the entire thing is going to look like.
 |
| Fig. 3. Mock-up 1 done by Umar |
 |
| Fig. 3.1. Mock-up 2 done by Umar for presentation |
we are visualizing of the actual place where we stand inside the VR. The user will be looking through 5 screens where the 3 screens will be showing the good and the other 2 will be showing the bad.
Finalizing our idea before presentation we decided that the goal of our project is to highlight society’s tendency to neglect social and environmental issues that are present within populated areas. The message behind this is that there’s always things beyond the positivity and perfect lives around you. It’s up to you to ignore or to actually care to take action. To be exact, our concept is showing the contrast of the good and bad side within the areas in Malaysia. We focuses on the popular places such as Sunway Pyramid Area, Ipoh and KL city. In order to show this environment, we decided to create a space as if the first person is in a room with a lot of windows around them. From this area we want to show an environment with 3 different part of the same area of the environment showing the good side while the other 2 showing the bad side. As for now, we are going to focus only to 1 particular place. However, as for the upscale, we are going to show 3 different popular place projecting the environment that has the positive and negative side.
The experience of this project will be as if the person is present and immerse with the environment around them with the ambisonic sound applied. The sound design will coincide with the motion tracking of the user, this will determine where sounds are being played. Whenever the user is looking at a ‘good’ screen the ‘good’ sounds play in the direction they are facing. Then for some time, the user starts to hear ‘bad’ sounds behind them which become more prevalent as the piece progresses. These sounds correlate with what the screens are showing at that point. The bad sounds such as people fighting or glass breaking is the visual cues for the person to look what is happening at their back. The bad visuals will be shown for a second and everything will go back to normal by having a fading effect.
For the execution of the prototype, an octagon room will be created, videos that are showing will be seen through windows and it is a flat video and embedded on the walls of the room. The room will stay as is throughout the entire experience. As for the execution for the upscaling part, videos will be placed outside the room, to form a parallax effect as the user explores the space. As the experience goes on, the room will begin to deteriorate, transforming from a nice room to a worn down place.
 |
| Fig. 3.2. Second Presentation on final concept |
Presentation Slide
As we presented, feedback and suggestion were given by the lecturers. One of the suggestion is to make the experiences be more interactive such as when the person is in the room, environmental issues were present such as trash are around them, the interaction here will ask the person to pick it up and throw it to the proper place which is the trash bin. However, if they ignore the trash that’s the time the room started to deteriorate. The more you ignore the situation around you, the more you will live in a miserable and uncomfortable environment.
11 April, 2019 (Week 2)
On the fourth day, we went for a trip to IPOH to get to know and explore about the heritage and culture of Malaysia. There were 4 stops we visited they are: Kinta Tin Mining (Gravel Pump), Perak famous Tanjung Tualang Tin Dredge Ship, Sekeping Kong Heng and the last stop is the Perak Cave Temple. All of these places helps on getting the information of the culture in IPOH. This trip showed me that some part of the places are clean and some places were filled with trashes that can be seen on the road. Going back to the project and correlating it, IPOH has a cleaner environment compared to Kuala Lumpur. This trip doesn’t have a big impact to the project that my group is creating. Not much information can be taken and to be used for the project building.
First Stop: Museum Perlombongan Bijih Timah Kinta
 |
| Fig. 4. Picture of Taylor's and RMIT students |
 |
| Fig. 4.1. Inside the museum perlombongan bijih timah kinta |
 |
| Fig. 4.2. Outside the museum perlombongan bijih timah kinta |
Second Stop: Tin Dredge Ship
 |
| Fig. 4.3. Things inside the tin dredge ship |
Third Stop: Sekeping Serendah for Lunch
 |
| Fig. 4.4. Some area of Sekeping Serendah Lane |
Forth Stop: Perak Cave Temple
 |
| Fig. 4.5. Inside and outside the cave temple |
12 April, 2019 (Week 2)
On the fifth day, we discussed all the things that we need to do for next week. We decided to get the footage in KL City and record the ambience around the environment. There will be 2 different content which will be the good side and the bad side of the city. We are planning to take a video of KL City buildings which has pleasant and decent environment and at the same time the area around it is surrounded by different environmental and social issues such as air pollution, water pollution, sound pollution, and many more that is happening in that area.
For the recording of the ambience, we listed all down focusing on the social and environmental audio such as people talking or shouting, car engines, honking, bottle breaking, spitting, insects buzzing and many more. These sounds will be the visual cues to trigger and direct the person to have a look at a glimpse of what is happening behind there.
 |
| Fig. 5. Some concept and ideas by Jaslyn |
15 April, 2019 (Week 3)
On the following week on Monday, we went to take and record all the footages and ambience sound of good and bad in Petaling Street. Footages for the bad side will be taken in the wet market, while the good side will be taken in the traditional Malay mall. Eight footages for both good and bad environment. Going back to campus by 4, we continued our research and progression. We were all busy with our own tasks; sound editing, video editing, 3D modeling and coding.
 |
| Fig. 6. Taking the footages in Pasar Seni |
 |
| Fig. 6.1. Taking the footages in Pasar Seni |
During the video shooting as well as audio recording, I also recorded some of the ambience sound and to be edited later on.
 |
| Fig. 6.2. Editing the ambience sound |
 |
| Fig. 6.3. Importing the footages taken from Pasar Seni into the model in Unity |
Everyone is staying late at night to figure out and make it work. Until then, Jaslyn manage to make the controller work.
 |
| Fig. 6.4. Controller is working to pick up an object |
16 April, 2019 (Week 3)
By Tuesday, we had our presentation for our progression. We showed the progression directly by experiencing directly with the use of the VR equipment without any presentation slides. We tested out the room and making sure the height and the position of the windows and everything that can be found in the room is properly positioned. As mentioned above the concept that we have, all 8 footages will be shown through the windows as the person enter in the VR space. However, some problems occur. We cannot insert more than 4 footages in the room because it lags and crashes at the same time due to the large size as we need to compress the file even more.
Some contribution that I have is that doing the coding part for head tracking called ray casting. However there’s a problem in the code that needs to tweak it out in order to make the code works. Progression wise, there is interaction of picking up stuff when the person is in the space already. Research for the coding is still going on. We were reminded by our lecturers that at least we are 95% done by Thursday figuring out the trouble shooting.
 |
| Fig. 7. Figuring out the trouble shooting |
 |
| Fig. 7.1. Keep on testing and adjusting |
 |
| Fig. 7.1. Keep on figuring and solving the problem |
17 April, 2019 (Week 3)
During our collaboration, we were trying to figure out the code for everything and it was so tough since we have no knowledge how to code in VR. Late in the evening, we seems to can’t figure it out some of the codes. As for that, we decided to change the concept with the same message that we want to deliver. Slight changes in inserting story and narration into the project. The very first concept is that when you are standing you will be seeing the good things but as you start turning your head back, the scenes started to morph and change into a bad environment view. It turns out that it is so difficult to code and that we are experiencing a lot of technical difficulties. Furthermore, our first concept has no strong narration since there is not much of interaction going on. Therefore, we changed our concept in order to meet the technical stuff and make the coding part less stressing. The new concept of our project now is that, the person will be starting off with good scene where there is a trash bin inside the room already. As you hit the trash can, the good footages shown will all change into the bad footages view surrounded with trashes around the floor. However, as you pick all the trash up, the bad footages will change into the previous normal scene with good footages shown. As for the upscaling part, we are going to make the transition between one scene to another even smoother.
 |
| Fig. 8. Trying out the HTC Vive |
18 April, 2019 (Week 3)
The last day before our final presentation, we decided to polish everything. Moreover, no sounds have been inserted at that time. For that I started to search some of the coding to input the sound when it collides with something. At first as I created the script, the code doesn’t work and I need to work on it. However, there is no much time left, we decided to ask help from our lecturer Mr. Razif.
 |
| Fig. 9. Coding the sound |
For now, we manage to code from looking at the good scene then when we hit something it changes to the bad scene. Also, there is interaction part, where people can pick up an object using the handle of the VR equipment. The part where picking up an object which is the trash to be put into the trash bin needs a code on detecting how many trash is inserted to the bin in order for the environment room to change into the good scene where currently the room is in a bad shape showing bad scene.
19 April, 2019 (Week 3)
Today was the final and the last day with the Australian students. At the same time, today was the day we need to present our final prototype to the panelist coming from outside Taylors. Before we presented later on in the afternoon, we still made some changes to the prototype such as putting the sound effect to the picking of an object. We also asked for our lecturer, Mr.Razif to help with the coding part. For the upscale we did a discussion on it and it was mentioned earlier we will be making the transition a lot smoother, adding some animation that is happening in the room, as well as play around with the sound intensity.
Upon our presentation going, some feedbacks were given directly from the panelist who is an expert in VR and has an experience on creating one. One of the panelist stated his opinion that what we are showing doesn’t show the full potential of making use of the VR technologies because we are just putting the person inside a room looking on some of footages around them. There is no feel of immersion in the prototype we are showing. In addition, the concept of picking up the trash is not relatable to the overall concept that we are trying to convey. For that, he suggested, why not bring the person in the VR into that particular environment which is the bad and the good places. Take them to that place to experience and feel the empathy towards the environment. From there may be picking up stuff can be applied to make the world a better place. Overall they like our ideation concept wide, however, we just need to execute and deliver it well in terms of using the technology VR wisely.
 |
| Fig. 10. Presenting to the Industry |
 |
| Fig. 10.1. The Industry people trying out the VR output |
As we were done with our presentation, some ideas came up to our mind and we discussed it with Mr.Razif. He as well gave his opinion for the upscale and for some changes to be made.
The suggestion he told us is that, we don’t need to change everything completely, the prototype itself can be the start of the journey. Since, when we enter the room, all we can see is a room with windows, it is suggested that the person can choose which windows they would like to enter to a particular environment. The windows represent a portal to another dimension. For that, we just need to think what are the things that triggers the bad things that we are on to change into the good ones. What can we do to make a difference.
Final Outcome
 |
| Fig. 10.2. A picture of everyone |
UPSCALE
6 May, 2019 (Week 6)
The upscale for this project, we decided to consider the panelist feedback of making the person in that good and bad environment instead of the person being in the room looking footages through the windows. Our idea for this is that we will be using a moving boat exploring the petaling street where there is good and the bad side. The environment will be moving like the idea of parallax. However, this idea is lacking the interaction. For that, this idea is to be confirmed.
We did a call to clarify and discuss on the upscaling part. We started listing down all the things, however, there were a lot of things that we didn't see it coming that we didn't consider.
 |
| Fig. 11. Video call through google hangout |
When we discussed our ideas to our lecturers, there are a lot of things that we need to consider especially when the time given is for 4 weeks. For that, we do another video call for another discussion. We decided to do our project on a smaller scale like a table top. However, after discussing it with the RMIT lecture, Dr, Li ping is against the idea as it was similar to another group. Therefore, we went with knee length height for the upscale.
 |
| Fig. 11.1. Few sketches for visualization |
13-22 May, 2019 (Week 7 - Week 8)
On the eight week, my group is still concern of pushing through this idea. However, when we asked advices from Mr. Mike and Mr. Kannan is that just follow what the Australian wants and after week 9, we will just come up with a new idea and concept. For now, just do what they ask and be done with it. We are pushing through with spawning the good houses on the ground and bad houses will appear at the same time. For the overlapping issue, we don’t mind as well as for the distance that the bad houses will randomly appear.
Mr. Mike advices us to create the prototype of this idea with the normal code that can be played using the mouse button. When everything works, then that’s the time we convert the code into VR. Converting the code will be done by the Australian that’s what we decided to do. I help in the coding of the UI making a radial menu. Then, I send the file to Jaslyn and she put everything together.
 |
| Fig. 12. creating radial button |
Our group decided to stick to the upscale plan and Jaslyn still proceed and do whatever she needs to do. She created a prototype where the model can snap into the grid.
 |
| Fig. 12.1. Snapping model to grid |
 |
| Fig. 12.2. Snapping model to grid and as it snaps another model appear |
Since the snapping of the model works already, Umar was in charge of switching the cubes into 3D model houses.
 |
| Fig. 12.3. changing the cube to houses |
 |
| Fig. 12.4. Prototype of the overall view done by Umar |
During this stage, this was all done in unity computer based. It has not been converted to VR. Therefore, when the coding is converted to VR, we don't whether the file is working or not because no hardware to test out the file.
29 May, 2019 (Week 9)
On the ninth week, we finished our tasked and send the code in google drive asking the Australian to test it out using VR equipment as we also code that is applicable for VR. However, any errors in the coding, we let the Australian figure it out and fix it since we code blindly and cannot test it out since no equipment. Any problem that they faced, we will still entertained and help them with the coding. Throughout the weeks before their presentation, Jaslyn keeps on helping to solve the problem however, no one is able to solve so they decided to use the prototype that was created during the 2 weeks time.
 |
| Fig. 13. RMIT Exhibition |
 |
| Fig. 13.1. RMIT Exhibition |
 |
| Fig. 13.2. RMIT Exhibition |
EXPANSION
27 May, 2019 - 12 July, 2019 (Week 9-15)
Through this expansion, I was able to explore into coding especially developing an augmented reality (AR) that I have never done and know in depth.
 |
| Fig. 14. Uploading image target in Vuforia |
 |
| Fig. 14.1. Creating the license key in Vuforia |
 |
| Fig. 14.2. Code for the UI button to function (scale up, down, next scene, left, right) |
 |
| Fig. 14.3. Code for linking to a website |
 |
| Fig. 14.4. First Attempt |
 |
| Fig. 14.4. Second Attempt using functional button |
 |
| Second Attempt Video |
Final Outcome:
 |
| Fig. 14.5. Introduction Page |
 |
| Fig. 14.6. Playing good and bad scene animation (AR) |
 |
| Fig. 14.6. function of button to go to a browser |
Group Design Report:
Individual Design Report:
Reflection:
This whole collaboration was a pressure in my part because there are so many things to be expected for the outcome. Everything in this PP1 was something new starting from collaborating with new people from different culture, exploring new technologies such as VR, and also getting things done in 2 weeks. Since it is too intensive, I am having a hard time understanding the code which is helped by my group member, Jaslyn. She is capable in solving and analyzing things quickly. I learnt so many things in all different aspects starting from voicing out opinions, brain storming, solving puzzles and most importantly team work. I can never thank enough my group members who supported me in all my difficulties especially in solving problem for coding as I listened and asked the things I don't understand.
Comments
Post a Comment