Add pabutools.recommendation: a recommendation system for participatory budgeting

docs-source/source/reference/index.rst

@@ -8,6 +8,7 @@ Reference
     election/index
     rules/index
     analysis/index
+    recommendation/index
     visualisation/index
     tiebreaking
     fractions

docs-source/source/reference/recommendation/index.rst

@@ -0,0 +1,16 @@
+Recommendation module
+=====================
+
+.. automodule:: pabutools.recommendation
+
+Sampling, pipeline and evaluation
+---------------------------------
+
+.. automodule:: pabutools.recommendation.recommendation
+   :members:
+
+Model training and prediction
+-----------------------------
+
+.. automodule:: pabutools.recommendation.model_training
+   :members:

docs-source/source/usage/index.rst

@@ -15,5 +15,6 @@ refer to the :ref:`quickstart` page.
     rules
     outcomevisualisation
     analysis
+    recommendation
     tiebreaking
     fractions

docs-source/source/usage/recommendation.rst

@@ -0,0 +1,189 @@
+Recommendation
+==============
+
+For reference, see the module :py:mod:`~pabutools.recommendation`.
+
+When a participatory budgeting instance has many projects, asking every voter about every
+project is a burden, and ballots are often left unfinished. This module implements the
+approach of Leibiker and Talmon (2023): ask each voter about only a few projects, estimate
+the votes that were never asked for, and run a voting rule on the completed profile.
+
+There are two ways to use it. If you are running a process and only hold partial ballots,
+:py:func:`~pabutools.recommendation.recommendation.elect` returns an outcome. If you hold
+complete ballots and want to know what asking less would have cost you,
+:py:func:`~pabutools.recommendation.recommendation.run_experiment` simulates the loss.
+
+Partial Ballots
+---------------
+
+A voter who was asked about only some of the projects has three kinds of opinion: projects
+she approved, projects she rejected, and projects she was never shown. This is stored in a
+:py:class:`~pabutools.election.ballot.cardinalballot.CardinalBallot` under the convention
+*positive means approved, negative means disapproved, zero or absent means not asked*.
+:py:func:`~pabutools.recommendation.recommendation.partial_ballot` builds one:
+
+.. code-block:: python
+
+    from pabutools.election import Instance, Project, ApprovalProfile, ApprovalBallot
+    from pabutools.recommendation import partial_ballot
+
+    garden = Project("Garden", 18)
+    crossings = Project("Crossings", 24)
+    library = Project("Library", 16)
+    shade = Project("Shade", 12)
+    instance = Instance([garden, crossings, library, shade], budget_limit=40)
+
+    # A voter who approved the garden, rejected the crossings, and was asked nothing else.
+    ballot = partial_ballot(approved={garden}, disapproved={crossings})
+
+Running a Process
+-----------------
+
+:py:func:`~pabutools.recommendation.recommendation.elect`
+
+Some voters do provide a complete ballot; they are the *learning voters*, and their profile
+is what the estimation learns from. Every other voter is asked about a limited number of
+projects.
+
+Which projects to ask about is decided by a *sampling setup*. The offline setups choose one
+set for everybody, out of the complete ballots already collected:
+
+.. code-block:: python
+
+    from pabutools.recommendation import offline_controversiality
+
+    lv_profile = ApprovalProfile([
+        ApprovalBallot([garden, library]),
+        ApprovalBallot([crossings, shade]),
+        ApprovalBallot([garden, shade]),
+        ApprovalBallot([garden, library, shade]),
+    ])
+
+    offline_controversiality(instance, lv_profile, k=2)
+    # {Library, Shade}  - the two projects the learning voters most disagree about
+
+The alternatives are :py:func:`~pabutools.recommendation.recommendation.random_setup`,
+:py:func:`~pabutools.recommendation.recommendation.offline_popularity` and
+:py:func:`~pabutools.recommendation.recommendation.offline_consensus`.
+
+Collect an answer for those projects from each remaining voter, and elect:
+
+.. code-block:: python
+
+    from pabutools.recommendation import elect
+
+    answers = {
+        "v5": partial_ballot(approved={garden}, disapproved={crossings}),
+        "v6": partial_ballot(approved={crossings}, disapproved={garden}),
+    }
+
+    elect(instance, lv_profile, answers, predictor="classification")
+    # BudgetAllocation([Garden, Shade])
+
+:py:func:`~pabutools.recommendation.recommendation.elect` returns a
+:py:class:`~pabutools.rules.budgetallocation.BudgetAllocation`, like any other rule. An
+answer a voter actually gave is never overwritten: the estimation only fills in the projects
+she was not asked about.
+
+To decide how many questions to ask in the first place,
+:py:func:`~pabutools.recommendation.recommendation.plan_sampling` turns a vote budget into a
+plan. Collecting 30% of the votes from 5000 voters over 100 projects, a tenth of them as
+complete ballots, means 150 people fill in the whole ballot and the rest answer 28 questions
+each:
+
+.. code-block:: python
+
+    from pabutools.recommendation import plan_sampling
+
+    plan_sampling(5000, 100, sample_degree=0.3, lv_degree=0.1)
+    # (150, 28)
+
+Asking Adaptively
+-----------------
+
+:py:func:`~pabutools.recommendation.recommendation.next_adaptive_question`
+
+The offline setups fix their questions in advance. The online setup cannot: it re-reads the
+tally after every answer, so each reply changes what is asked next. A live process drives it
+one question at a time:
+
+.. code-block:: python
+
+    from pabutools.recommendation import next_adaptive_question, partial_ballot
+
+    answers = partial_ballot()
+    for _ in range(2):
+        project = next_adaptive_question(instance, lv_profile, answers)
+        # ... put `project` to the voter, then fold her reply into `answers` ...
+
+Estimating the Missing Votes
+----------------------------
+
+Three prediction modules are provided:
+:py:func:`~pabutools.recommendation.model_training.predict_by_classification` (one binary
+classifier per project),
+:py:func:`~pabutools.recommendation.model_training.predict_by_matrix_factorization` and
+:py:func:`~pabutools.recommendation.model_training.predict_by_factorization_machines`
+(collaborative filtering).
+
+They are selected by name, as in the ``predictor="classification"`` argument above. Any
+callable with the same signature works, so you can supply your own:
+
+.. code-block:: python
+
+    def approve_everything(instance, lv_profile, partial_ballots):
+        return {voter: ApprovalBallot(instance) for voter in partial_ballots}
+
+    elect(instance, lv_profile, answers, predictor=approve_everything)
+
+These three modules rely on ``xgboost``, ``scikit-surprise`` and ``lightfm``, which are
+**not** installed with Pabutools. They come with the optional ``recommendation`` extra::
+
+    pip install pabutools[recommendation]
+
+The libraries are imported only when a model is actually fitted, so everything else in this
+module works without them.
+
+Measuring the Loss
+------------------
+
+:py:func:`~pabutools.recommendation.recommendation.run_experiment`
+
+If you already hold complete ballots, you can measure what asking less would have cost. This
+is a simulation: the profile is split into the voters who answer in full and the ones who do
+not, each of the latter has her answers to k projects read off her real ballot, and the rest
+are withheld for the prediction module to estimate. The allocation that comes back can then
+be compared with the one complete information produces:
+
+.. code-block:: python
+
+    from pabutools.recommendation import (
+        run_experiment, greedy_approval, fractional_allocation_score,
+    )
+
+    profile = ApprovalProfile([
+        ApprovalBallot([garden, library]), ApprovalBallot([crossings, shade]),
+        ApprovalBallot([garden, shade]), ApprovalBallot([garden, library, shade]),
+        ApprovalBallot([crossings, shade]), ApprovalBallot([garden, library]),
+    ])
+
+    real = set(greedy_approval(instance, profile))
+    predicted = set(run_experiment(
+        instance, profile, 0.5, 0.5, setup="offline_controversiality", seed=17
+    ))
+
+    fractional_allocation_score(real, predicted, instance.budget_limit)  # 0.75
+    len(real ^ predicted)                                                # 0
+
+The two arguments ``0.5, 0.5`` are the *sample degree* — the share of all voter-project
+votes that is collected — and the *LV degree*, the share of those votes coming from complete
+ballots. :py:func:`~pabutools.recommendation.recommendation.fractional_allocation_score`
+reports the cost of the correctly predicted projects as a share of the budget, and the
+symmetric difference counts the projects on which the two allocations disagree.
+
+:py:func:`~pabutools.recommendation.recommendation.run_all_experiments` repeats this over a
+grid of both degrees, for every setup and every prediction module at once.
+
+To score the predicted votes themselves rather than the allocation, use
+:py:func:`~pabutools.recommendation.recommendation.classification_metrics`, which reports
+precision, recall and F1 over the projects a voter was never asked about.

pabutools/recommendation/__init__.py

@@ -0,0 +1,109 @@
+"""
+Module implementing the algorithms of
+"A Recommendation System for Participatory Budgeting",
+by Gil Leibiker and Nimrod Talmon (2023), https://optlearnmas23.github.io/files/p17.pdf
+
+:py:mod:`~pabutools.recommendation.recommendation` holds the sampling setups,
+the pipeline, the voting rule and the evaluation metrics;
+:py:mod:`~pabutools.recommendation.model_training` holds the fitting and
+prediction of the three learning-based modules (whose ML libraries are optional
+dependencies, installed with ``pip install pabutools[recommendation]``).
+
+Programmer: Roei Yanku
+"""
+
+from pabutools.recommendation.recommendation import (
+    APPROVAL,
+    DISAPPROVAL,
+    HIDDEN,
+    partial_ballot,
+    reveal_ballot,
+    approved_projects,
+    disapproved_projects,
+    exposed_projects,
+    hidden_projects,
+    as_approval_ballot,
+    consensus_levels,
+    most_consensual_projects,
+    greedy_approval,
+    random_setup,
+    offline_popularity,
+    offline_consensus,
+    offline_controversiality,
+    online_adaptive_controversial,
+    next_adaptive_question,
+    SETUPS,
+    PREDICTORS,
+    exposed_sets,
+    plan_sampling,
+    split_lv_tv,
+    complete_ballots,
+    run_pipeline,
+    run_experiment,
+    Predictor,
+    as_predictor,
+    elect,
+    SAMPLE_DEGREES,
+    LV_DEGREES,
+    run_all_experiments,
+    classification_metrics,
+    fractional_allocation_score,
+)
+from pabutools.recommendation.model_training import (
+    train_classification,
+    train_matrix_factorization,
+    train_factorization_machines,
+    predict_by_classification,
+    predict_by_matrix_factorization,
+    predict_by_factorization_machines,
+)
+
+__all__ = [
+    # Section 2.3 - partial ballots and the +1/-1/0 score convention.
+    "APPROVAL",
+    "DISAPPROVAL",
+    "HIDDEN",
+    "partial_ballot",
+    "reveal_ballot",
+    "approved_projects",
+    "disapproved_projects",
+    "exposed_projects",
+    "hidden_projects",
+    "as_approval_ballot",
+    # Section 2.2 - popularity, consensus and the voting rule.
+    "consensus_levels",
+    "most_consensual_projects",
+    "greedy_approval",
+    # Section 3.1 - the sampling setups.
+    "SETUPS",
+    "random_setup",
+    "offline_popularity",
+    "offline_consensus",
+    "offline_controversiality",
+    "online_adaptive_controversial",
+    "next_adaptive_question",
+    "exposed_sets",
+    # Section 2.1 - the prediction modules.
+    "PREDICTORS",
+    "Predictor",
+    "as_predictor",
+    "train_classification",
+    "train_matrix_factorization",
+    "train_factorization_machines",
+    "predict_by_classification",
+    "predict_by_matrix_factorization",
+    "predict_by_factorization_machines",
+    # Running a process: a real one, or the paper's experiments.
+    "plan_sampling",
+    "elect",
+    "split_lv_tv",
+    "complete_ballots",
+    "run_pipeline",
+    "run_experiment",
+    "run_all_experiments",
+    "SAMPLE_DEGREES",
+    "LV_DEGREES",
+    # Section 5 - evaluation.
+    "classification_metrics",
+    "fractional_allocation_score",
+]