Web Scraping Practice

Assignment Ideas

Here are some assignment ideas to get you started. You can work on one of these ideas, or pick something entirely different.

  1. Dataset of Quotes (BrainyQuote): Create a dataset of quotes for different tags/topics by scraping the site:https://www.brainyquote.com/topics

  2. Dataset of Movies (TMDb): The Movie Database (TMDb) contains information about thousands of movies from around the world: https://www.themoviedb.org/movie. Can you scape the site to create a dataset of movies containing information like title, release date, cast, etc. ? You can also create datasets of movie actors/actresses/directors using this site.

  3. Dataset of TV Shows (TMDb): The Movie Database (TMDb) contains information about thousands of TV shows from around the world: https://www.themoviedb.org/tv. Can you scape the site to create a dataset of TV shows containing information like title, release date, cast, crew, etc. ? You can also create datasets of TV actors/actresses/directors using this site.

  4. Dataset of Books (BooksToScrape): Create a dataset of popular books in different genres by scraping the site Books To Scrape: http://books.toscrape.com

  5. Dataset of Quotes (QuotesToScrape): Create a dataset of popular quotes for different tags by scraping the site Quotes To Scrape: http://quotes.toscrape.com

  6. Scrape User's Reviews (ConsumerAffairs): ConsumerAffairs contains reviews about thousands of brands: https://www.consumeraffairs.com/. Can you scrape any category from the site to create a dataset of Reviews containing information like Title, Rating, Reviews and toll-free numbers, etc.?

  1. Songs Dataset (AZLyrics): Create a dataset of songs by scraping AZLyrics: https://www.azlyrics.com/f.html. Capture information like song title, artist name, year of release, and lyrics URL.

  2. Scrape a Popular Blog: Create a dataset of blog posts on a popular blog e.g. https://m.signalvnoise.com/search/. The dataset can contain information like the blog title, published date, tags, author, link to the blog post, etc.

  3. Weekly Top Songs (Top 40 Weekly): Create a dataset of the top 40 songs of each week in a given year by scraping the site https://top40weekly.com. Capture information like song title, artist, weekly rank, etc.

Also check out these projects and tutorials: