Data Scraping Basics - course

A practical introduction to collecting and pre-processing textual data in Python, in the context of digital humanities.

API keys setup (Genius, Europeana etc.)

Create a .env file (use the .dev.env as template) at the root of the project:

GENIUS_CLIENT_ACCESS_TOKEN=your_client_access_token_here
WSKEY=your_europeana_api_key

This file is ignored by git and must never be committed. See this guide to get your key.

Installation

With pip + venv

python3.11 -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -r requirements.txt

With uv

uv sync

With conda

conda env create -f environment.yml
conda activate sassari

=======

With pyenv

pyenv install 3.11
pyenv virtualenv 3.11 sassari
pyenv activate sassari
pip install -r requirements.txt

Launch JupyterLab

# venv or conda or pyenv (environment already activated)
jupyter lab

# uv
uv run jupyter lab

Then open the desired notebook from the notebooks/ folder.

Build Jupyter Book

jupyter book start 
jupyter book build --html

Citation

@misc{jolivet2024datascrapingprocessing,
  author       = {Jolivet, Vincent},
  contributor  = {Conte, Lilla and Terriel, Lucas and Popineau, Maxime},
  title        = {Data scraping and (pre)processing course},
  year         = {2024-2026},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/architexte/cours-data-processing}},
  institution  = {École nationale des chartes - PSL},
  note         = {Course materials, École nationale des chartes}
}