This blog post intends to describe the process I went through to create compareyourlife.com. I’ve stripped out most of the technical jargon in order to make it readable for anyone who is interested in how this site was created.
'API' is the only technical acronym I have used in this post. It stands for Application Programming Interface and, for the purposes of this project, you can think of it as a “toolset” that compareyourlife.com uses to programmatically access content from another website.
The Aim
I wanted a small project that allowed me to deepen my knowledge of the latest web technologies and cloud platforms. I wanted to develop a website that was interesting and/or helpful to people in some unique way. I wanted it to be fully automated and API driven, requiring no content generation or manual effort from myself to keep it updated or functioning.
The result was compareyourlife.com. This site utilises Wikipedia's API (MediaWiki) to automatically collate and display information about historical events relating to individuals at their respective ages. You can search by your birthday to find out what other notable people in history were doing at your current age.
Some would say that you should never compare your life to others. It turns out that it’s actually pretty interesting. I’m similar in age to Napoleon Bonaparte when he was proclaimed Emperor of France. I’m also similar in age to Muhammed Ali when he defeated Joe Frazier in the ‘Thrilla in Manila’. It certainly puts your life into perspective!
The Content
Where could I get all this information? Wikipedia of course. And wouldn’t you know it, Wikipedia has an open API available which provides access to all of its content. Great stuff!
I then had to decide where in Wikipedia to get the content from. At first, I thought Wikipedia’s Current Events Portal would be the ideal place to start. After some detailed analysis, I decided this portal contained way too much information about events that occur each and every day. How was I ever going to filter this content to something the average person would find interesting?
So, my next port of call was Wikipedia’s Selected Anniversaries/On This Day archive . If you take a look at Wikipedia’s homepage you’ll see a section that lists important events on this day:
https://en.wikipedia.org/wiki/Main_Page
A key requirement for an event to be listed in this section is:
“The event needs to be of moderate to great historical significance (relative to the other historical events that occurred on the same day of the year). As much as possible, the array of topics should include a variety of years (e.g., not just limited to the 20th/21st centuries), geographical areas (e.g., more than just the English-speaking world), and subjects (e.g., not too many articles on war or technology).”
It turns out you can access Selected Anniversaries/On This Day events for any calendar day in the year. And the Selected Anniversaries pages link to Actual Day pages themselves which again only seem to contain significant historical events. This was exactly what I required so I decided to use these pages for the underlying event data. Here are some examples of the the pages:
https://en.wikipedia.org/wiki/Wikipedia:Selected_anniversaries/August_31
https://en.wikipedia.org/wiki/August_31
So, with all this interesting information at hand it was time to start thinking about how to collect the events I required.
Retrieving The Events
The website needed to retrieve an event from Wikipedia, check if it related to a person, if so then retrieve that person’s birthday, calculate their age at the time of the event, and ultimately display this event when a similar aged person searched their birthday on the website.
To retrieve the events I used Wikipedia’s API to retrieve the relevant sections of text from the individual Selected Anniversaries / Actual Day pages, i.e.
---------------------------------------------------------------------------------------------------------------------------
August 31: Day of Arafah (Islam, 2017)
Interior view of Kinetoscope with peephole viewer
1897 – Thomas Edison was granted a patent for the Kinetoscope (pictured), a precursor to the movie projector.
1939 – Nazi forces, posing as Poles, staged an attack against the German radio station Sender Gleiwitz in Gleiwitz, Upper Silesia, Germany, creating an excuse to invade Poland the next day.
1969 – Approximately 150,000 people attended the Isle of Wight Festival, which featured Bob Dylan in his first gig since 1966.
1982 – Anti-government demonstrations were held in 66 Polish cities to commemorate the second anniversary of the Gdańsk Agreement, which allowed the Solidarity trade union to be established.
1997 – Diana, Princess of Wales, her companion Dodi Fayed, and their driver Henri Paul were killed in a high speed car accident in the Pont de l'Alma road tunnel in Paris.
Aidan of Lindisfarne (d. 651) · Josephine St. Pierre Ruffin (b. 1842) · Sara Ramirez (b. 1975)
More anniversaries:
---------------------------------------------------------------------------------------------------------------------------
Once I had the above, I utilised some string methods to keep only elements (or lines) which had a hyphen “ – “ in them and discarded everything else. This returned only the events:
---------------------------------------------------------------------------------------------------------------------------
1897 – Thomas Edison was granted a patent for the Kinetoscope (pictured), a precursor to the movie projector.
1939 – Nazi forces, posing as Poles, staged an attack against the German radio station Sender Gleiwitz in Gleiwitz, Upper Silesia, Germany, creating an excuse to invade Poland the next day.
1969 – Approximately 150,000 people attended the Isle of Wight Festival, which featured Bob Dylan in his first gig since 1966.
1982 – Anti-government demonstrations were held in 66 Polish cities to commemorate the second anniversary of the Gdańsk Agreement, which allowed the Solidarity trade union to be established.
1997 – Diana, Princess of Wales, her companion Dodi Fayed, and their driver Henri Paul were killed in a high speed car accident in the Pont de l'Alma road tunnel in Paris.
---------------------------------------------------------------------------------------------------------------------------
The next step was to identify which of these events related to a person, so I again used the Wikipedia API to cycle through each individual hyperlink in every event to find out if the related Wikipedia page had an element labelled ‘bday’ in it and if so I returned the contents of that bday element (essentially I was searching for a class name for those who know CSS – i.e. <span class="bday">1961-07-01</span>). This returned the following array (or list):
---------------------------------------------------------------------------------------------------------------------------
https://en.wikipedia.org/wiki/Thomas_Edison 1847-02-11
https://en.wikipedia.org/wiki/Bob_Dylan 1941-05-24
https://en.wikipedia.org/wiki/Dodi_Fayed 1955-04-15
https://en.wikipedia.org/wiki/Henri_Paul 1956-07-03
---------------------------------------------------------------------------------------------------------------------------
From the above information, I could deduce that only 3 of the events on that page related to a person (or several people). If I couldn't find any bday elements in any of the links relating to an event, I assumed the event was not related to a person. I used the retrieved birthday’s to calculate each person’s age and then stored the results in a database for future reference. The database stores similar information to the below:
---------------------------------------------------------------------------------------------------------------------------
Event 1: 1897 – Thomas Edison was granted a patent for the Kinetoscope (pictured),
a precursor to the movie projector.
Name: Thomas Edison
Born: 1847-02-11
Age when this event occurred: 50 years, 6 months and 20 Days old.
Event 2: 1969 – Approximately 150,000 people attended the Isle of Wight Festival, which
featured Bob Dylan in his first gig since 1966.
Name: Bob Dylan
Born: 1941-05-24
Age when this event occurred: 28 years, 3 months and 7 days old.
Event 3: 1997 – Diana, Princess of Wales, her companion Dodi Fayed, and their driver
Henri Paul were killed in a high speed car accident in the Pont de l'Alma road tunnel in
Paris.
Name: Diana, Princess of Wales
Born: 1961-07-01
Age when this event occurred: 36 years, 1 month and 30 days
Event 4: 1997 – Diana, Princess of Wales, her companion Dodi Fayed, and their driver
Henri Paul were killed in a high speed car accident in the Pont de l'Alma road tunnel in
Paris.
Name: Dodi Fayed
Born: 1955-04-15
Age when this event occurred: 42 years, 4 months and 16 days
Event 5: 1997 – Diana, Princess of Wales, her companion Dodi Fayed, and their driver
Henri Paul were killed in a high speed car accident in the Pont de l'Alma road tunnel in
Paris.
Name: Henri Paul
Born: 1956-07-03
Age when this event occurred: 41 years, 1 month and 28 days
---------------------------------------------------------------------------------------------------------------------------
So, by now you can see that it took 20+ API calls just to retrieve the required event information from Wikipedia for only a single Selected Anniversaries page. I had 365 more Selected Anniversaries pages to loop through PLUS another 366 Actual Day pages – each containing 50 to 100+ events themselves. If you do the maths, that’s a lot of API calls – Upwards of 40,000 API calls.
I quickly realised this was not something that could be run every single time that someone searched their birthday on compareyourlife.com. The only sensible way to proceed was to run it once and have it populate a back-end database which is then queried by the website. Then all I need to do is periodically update that database so that recent events are included in the search results.
The current database is up to date as of August 16th 2017. I have created a separate application to run additional API calls and append more recent events to the database.
So it is still automated in a sense, it just requires a little manual intervention every now and again.
Creating The Website
I ended up using Wix Code to host the website and back-end database:
"Wix Code makes it easy to build web applications and robust websites. It’s serverless, hassle-free coding. Set up database collections, create content rich websites, add custom forms and change site behavior with our APIs. All built with the stunning visual components of the Wix Editor. Plus, anything you create is SEO compatible."
One day I'll wrote a blog post about my experiences with Wix Code. For now I'll just say that overall it was a very positive experience and it allowed me to get a fully functioning website (with back-end database) up in running in a matter of minutes.
That's All Folks
And that’s the making of compareyourlife.com - a website that allows you to search your birthday to find out what other notable people in history were doing at your current age. Thank you to Wikipedia for all the event data!
Comments