Boss Level 1 — CLEARED
Going into this program, I was already having doubts about my abilities and just constantly waiting for the days to past until I can finally be free. After dipping my feet in for three weeks, I realized that with a program as fast paced as this one, the day of graduation will come much sooner than I see it. However, I still must always have my head on straight. I need to keep up with my assignments, and make sure I am up to par with the content.
A program like this can seem like a fun challenge to adults, who have already had their taste of the real world. But being a 19-year old temporary college-dropout who is now coming to full terms with their depression, this feels like I am my own player in this very complex video game. However, I already am aware to some of the many obstacles in this game so clearing this level won’t be so bad, yeah?
Coming into this project, I was nervous as hell. I had absolutely no idea where to start and I was not fully understanding the concepts of using an API nor scraping. At first I planned on using an API since most of the information I needed will be stored in the API; I would just have to worry about what specific information I needed to use. The next lecture was then about scraping, and how to extract simple information on a website. To be honest,I did not understand the lecture the first time around, but after watching it a couple more times, I was able to get the hang of how to scrape websites and how to scrape the information I need, therefore, I changed my projected from being an API-based, to scraping.
Since I still am not 100% with how scraping works, I decided to do something very simple. I was going to find a website on a favorite cartoon of mine (Winx Club. An underrated classic.) and scrape around two or three attributes from the main protagonists, the Winx. It’s safe to say I took the easy train, simply because I was too scared to do anything to the extreme because I did not want to add on to the already applied stress I had when starting the project. This is where the funny part comes.
You would think a simple WikiFandom website would not be that difficult to scrape, that finding the attributes you want can be a piece of cake. God, I was so wrong. The depths of the website I used was so deep, I couldn’t even find the html for just the characters. It was nested deep into multiple tables and collapsible tables that a simple alias was buried from a hash into another hash, into another. So I tried a much simpler website, with just the main protagonists and the main sub-characters.
Thankfully, this website was not as complex as the previous one, but it was still very difficult to extract the most simplest of attributes. The main attribute I wanted was their powers and abilities. However, they are nested into a collapsible folder, or in English, an array buried inside a hash. In the display website, I need to click a button that would then drop down a table, displaying their abilities. The html doesn’t look very complex, but the extraction of those attributes definitely was.
Finally, after prying for a good hour and a half, I was able to find the html for the powers and abilities for the main protagonists.
I didn’t want my application to be too boring so I also decided to add their alias. This was also a difficult task, since the location of a few characters’ alias was different. This meant having to form a couple if-statements for the circumstance in which the user chooses one them.
Example:
Alias for Tecna = doc.css(“td”)[5].text.strip
Alias for Musa = doc.css(“td”)[6].text.strip
Believe it or not, this experience was quite amusing, especially when looking at my sister’s project. Her application was much more dense and detailed, giving the user many other options of action, such as finding the location of an artist, their background, what gallery they are linked with, etc. I was amazed by how advanced her application was, confused why it was so difficult for me to obtain just a couple attributes. She admitted that the website she was scraping from was very simple and neatly organized that it wasn’t hard to find anything at all. It’s really funny and beyond ironic when I think about it, how I viewed this program, especially this project, with a reluctant mindset, and using a very difficult website for my project, and my sister who has a more optimistic mentality and getting a much easier and cleaner source. Part of me likes to think it was karma for being so negative about this program, or maybe it was God trying to test me and show me what I can be capable of.
Overall, this project was really interesting and makes me curious to learn about the next one. It’s like the adrenaline when you defeated the first boss level in the video game. It gives you the confidence to defeat the next boss, and the next, so on so forth.
