Web Scraping and Data Mining Mastery with Python Online Course

Web Scraping and Data Mining Mastery with Python Online Course

This course, Data Scraping and Data Mining from Beginner to Professional, teaches you how to extract and work with data from the Internet using Python. Designed for beginners, it starts with fundamentals and gradually moves to advanced practical implementations, reinforced with live coding, quizzes, and detailed explanations. The course features four hands-on projects that provide experiential learning, helping you understand the gap between theory and practice and develop in-demand data scraping and mining skills for the workplace.

Who should take this Course?

The Web Scraping and Data Mining Mastery with Python Online Course is ideal for data analysts, Python developers, and aspiring data scientists who want to efficiently collect, process, and analyze data from the web. It is also valuable for students, researchers, and professionals in business, marketing, and finance who wish to gain hands-on skills in extracting actionable insights from large datasets using Python-based web scraping and data mining techniques.

What you will learn

• Understand the difference between synchronous and asynchronous requests
• Apply BS4 for parsing the response data from the server
• Explore the different tools that are used for data scraping; namely, Requests, BS4, Scrapy, Selenium
• Understand BS4 parser functions for getting the data out of the HTML
• Learn to use Scrapy to write the spiders for crawling websites and extracting data
• Learn to use Selenium to understand the automation and control of web flows

Introduction

• Why Data Scraping
• Applications of Data Scraping
• Introduction of Instructor
• Introduction to Course, Scraping, Tools
• Projects Overview

Requests

• Introduction to Python Requests
• Hand on with Requests
• Extracting Quotes Manually
• Quiz (Extracting Authors)
• Solution (Extracting Authors)
• Pagination
• Quiz ( Extracting Author and Quotes)
• Solution 01 (Extracting Author and Quotes)
• Solution 02 (Extracting Author and Quotes)
• Ajax Requests
• Ajax Requests for Cricket Information
• Ajax Requests Pagination
• Quiz (Extracting Top Stats from Cricket info)
• Solution 01 (Extracting Top Stats from Cricket Information)
• Solution 02 (Extracting Top Stats from Cricket Information)

Beautiful Soap 4 (BS4)

• Introduction to BS4
• Quiz (Difference Between Requests and BS4)
• Solution (Difference Between Requests and BS4)
• Hands-On with BS4
• Extracting Data from Tree
• Extracting Quotes from the Website
• Quiz (Extracting Author Names)
• Solution (Extracting Author Names)
• Attributes of Tags in BS4
• Multi-Valued Attributes of Tags in BS4
• Scraping Movie Names from IMDB
• Quiz (Getting the Ratings, Year, Name of the Movie)
• Solution 01 (Getting the Ratings, Year, Name of the Movie)
• Solution 02(Getting the Ratings, Year, Name of the Movie)
• Scraping Time, Genre, and Release Date from IMDB 01
• Scraping Time, Genre, and Release Date from IMDB 02
• Combining Two Requests Data for IMDB
• Movies Recommender System (Creating Movie URL)
• Movies Recommender System (Creating Director URL)
• Movies Recommender System using BS4 (Getting Top 4 Movies)
• Movies Recommender System using BS4 (Merge All Requests Together)

CSS Selectors

• Introduction to CSS Selectors
• CSS Selectors Hands-On (Tags)
• Quiz (Tags)
• Solution (Tags)
• CSS Selectors Hands-On (Descendants, ID, Class)
• Quiz (Descendants)
• Solution (Descendants)
• Quiz (ID)
• Solution (ID)
• Quiz (Class)
• Solution (Class)
• CSS Selectors Hands-On (Nested Tags, ID Tags, Class Tags)
• Quiz (Class with Tag)
• Solution (Class with Tag)
• CSS Selectors Hands-on(Coma Separator, Universal Selectors
• Quiz (Combining Two Selectors)
• Solution (Combining Two Selectors)
• CSS Selectors Hands-On (Sibling Notations and Direct Child)
• Quiz (Adjacent Sibling)
• Solution (Adjacent Sibling)
• Quiz (General Sibling)
• Solution (General Sibling)
• CSS Selectors Hands-On (Child Selectors)
• Quiz (First Child)
• Solution (First Child)
• Quiz (Only Child)
• Solution (Only Child)
• Quiz (Last Child)
• Solution (Last Child)
• CSS Selectors Hands-On (Negations, Attributes)
• Quiz (Negation)
• Solution (Negation)
• CSS Selectors Hands-On (Attributes, Attribute Values)
• Quiz (Attribute Values)
• Solution (Attribute Values)
• CSS Selectors Hands-On (Attributes Wild Cards Values)
• Quiz (Attributes Wild Card)
• Solution (Attributes Wild Card)

Scrapy

• Introduction to Scrapy
• Comparison of Scrapy and Requests
• Scrapy at a Glance Documentation
• Getting Started with Scrapy
• Running Documentation Spider 1
• Running Documentation Spider 2
• Writing Spider from the Scratch
• Understanding the Response (URL, Status)
• Understanding the Response (Headers)
• Understanding the Response (Values in Headers)
• Understanding the Response (Body)
• Understanding the Response (Request)
• Understanding the Response (Meta)
• Understanding the Response (Flags, Certificate, ip_address, Copy)
• Understanding the Response (replace, urljoin, follow, follow_all)
• Response CSS and Scrapy Shell
• Extracting Quotes
• Understanding Nested Selectors
• Extracting the Author and Quotes
• Checking for Next Page
• Checking for Next Page in Spider
• Checking for Next Page URL
• Scraping Quotes from Next Pages
• Exporting Extracted Data
• Quiz (Get the Tags)
• Solution (Get the Tags)
• Next Website
• CSS Selectors for Movie Names and URLs
• Combined CSS Selectors for Movie Names and URLs
• Send Request to the Film Information Page
• Merge Data from Two Callbacks
• Extracting Movie Duration and Genres
• Exporting the Extracted Data
• Quiz (Extracting the Year)
• Solution (Extracting the Year)
• Getting Director Name and URL
• Getting Top Four Movies of Directors
• Extracting Data
• Extracting Data Anomaly (CSS Selector)
• Extracting Data Anomaly (dont_filter Flag)

Scrapy Project

• Hugoboss Website for Scraping
• Understanding Site Structure
• Writing CSS Selectors for Listings
• Listings in Scrapy Shell
• Sending Request to Listings URLs
• Writing CSS for Getting the Product from the listings
• Extracting Products URL from the Listings
• Sending Requests to Products of the Listings
• Writing CSS for Getting the Product Information
• Getting the Bigger Images of the Product
• Adding Pagination to Spider and Running It
• Output of the Spider

Selenium

• Introduction to Selenium
• Getting Started with Selenium
• Configuring the Webdriver
• Extracting Quotes
• Extracting Quotes and Author Names
• Quiz (Extracting Quotes)
• Solution (Extracting Quotes)
• Clicking on Button
• Pagination and Extracting Data
• Exception Handling for Unavailable Elements
• Navigating the Website for Login
• Quiz (Log In and Extract Quote)
• Solution (Log In and Extract Quote)

Project Selenium

• Overview of Project
• Closing the Cookie Button
• Setting the Language for Translation
• Sending the Text for Translation
• Downloading the Translation
• Reading Data from File for Translation