Skip to content

Technologies used for web-scraping  #75

Description

@jasmineyang

We want a simple beginner's tutorial to have the installation steps for basic technologies used for web-scraping. The goal is that students can become acquainted with these tools (e.g. beautifulsoup, Scrapy) by just following the exact steps, and then later on we may use examples (#70) to demonstrate the technologies.

List of topics in order:

  • Introduction to web-scraping: What it is and why is it useful
  • BeautifulSoup: Installation, Expressions & Examples (eg. extracting needed information from HTMP pages)
  • Manually scape data using browser extensions
  • Scrapy: Installation, Rules & Examples (eg. writing a simple scraper, telling Scrapy to follow URLs and scrape contents)
    and more.

Link to notes: https://github.com/ubcecon/computing_and_datascience/blob/master/python_sandbox/Web-Scraping.md

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions