ER Diagram of information in PLDB

[kaby76]

It's not complete, but here is an ER diagram of the information in PLDB.

PLDB Entity Descriptions

Each section below describes one entity (or logical group) shown in PLDB_Overview.png.

---

Storage model — two-tier

Tier	Entities	Source files	Export
First-class files	Concept, Creator, Book	concepts/*.scroll, creators/creators.scroll, books/books.scroll	pldb.json, creators/creators.json, books/books.json
Embedded value objects	GithubRepo, Wikipedia, Tiobe, …	Sub-trees inside a Concept's .scroll file	Flattened into pldb.json with _ separator (e.g. githubRepo_stars)

Value objects have no files of their own; they are sub-documents of the Concept that owns them.

---

Concept

Source: concepts/<id>.scroll (one file per concept)
Export: pldb.json (~5,146 records as of last build)

The central entity of the entire database. One record per "thing" that PLDB tracks — this includes not just programming languages but also compilers, interpreters, data formats, protocols, editors, query languages, standards, virtual machines, and more. The tags field classifies which type each concept is (e.g. pl, compiler, queryLanguage, xmlFormat).

Every other entity in the diagram either belongs to a Concept (embedded value objects), describes a Concept (Creator, Book), or links Concepts together (ConceptRelationship).

Key fields:

Field	Notes
id	Slug used as the filename, e.g. python, cpp
name	Display name
appeared	Year of first public release/announcement (required)
tags	One or more type tags from a controlled vocabulary of ~80 values
creators	Space-delimited names, cross-referencing the Creator entity
lab	Community or organization where the language originated
country	Country of origin
website, spec, download, docs	Official URLs
fileExtensions	File extensions recognized for this language
isOpenSource, isPublicDomain	Licensing flags
rank, pldbScore, foundationScore	Computed popularity/importance metrics
isLanguage	Computed boolean — is this concept classified as a language?
isSelfHosted	Computed — is the main implementation written in itself?

---

Creator

Source: creators/creators.scroll
Export: creators/creators.json (~370 records)

A curated directory of people who created or significantly designed programming languages. This is a hand-maintained list, not scraped. It is linked to Concept via the creators field on Concept, which stores names as a delimited string — there is no enforced foreign key.

Field	Notes
name	Full name, used as the identifier and as the match target in Concept.creators
born, died	Birth/death years
country	Country of origin
homepage	Personal or professional website
gits	GitHub or GitLab profile URL
twitter	Twitter/X profile URL
pldbInterview	Link to an interview published on PLDB
wikipedia	Link to their Wikipedia page
primaryLink	Computed — first populated of: homepage, gits, twitter, wikipedia, pldbInterview

---

Book

Source: books/books.scroll
Export: books/books.json (~100 records)

A curated reading list of notable programming and computer science books — titles like The C Programming Language, Design Patterns, Clean Code, and The Art of Computer Programming. This is an editorially selected list, not a scraped catalogue.

The languagesUsed field is a space-separated string of Concept IDs (e.g. "cpp smalltalk"), making it the only link from Book back to Concept. This is a loose reference — no FK constraint is enforced in the source format.

Do not confuse with the goodreads and isbndb crawled blobs embedded inside Concept. Those are scraped from external sites and contain books written about a specific language. The Book entity is an independent, first-party curated list.

Also note: Concept.bookCount (a computed metric) is derived from the goodreads/isbndb crawl data — not from this Book entity.

Field	Notes
bookTitle	Title, used as identifier
authors	Author name(s), delimited by " and "
year	Publication year
publisher	Publisher name
pages	Page count
homepage	URL to book website or Wikipedia page
pdf	URL to a freely available PDF, if one exists
source	URL to the source code accompanying the book
languagesUsed	Space-separated Concept IDs for languages featured in the book

---

FeatureFlags

Not a separate file. Boolean attributes stored directly on the Concept's .scroll file and flattened into pldb.json as top-level keys.

Each of the ~100 feature flags answers a yes/no question about a language's design. All extend abstractFeatureParser, which means each flag:

• Has a bool value (true / false / absent = unknown)
• May carry an indented code example demonstrating the feature
• Has a human-readable title and optional pseudoExample and reference URL in the schema

The flags drive the generated features/<featureName>.html pages, which show all languages that have or lack a given feature.

Examples of feature flags and what they test:

Flag	Question
hasClasses	Does the language have a concept of classes?
hasGarbageCollection	Does the language have a garbage collector?
hasStaticTyping	Does the language have static typing at compile time?
hasGenerics	Does the language have generics?
hasSemanticIndentation	Does the language use indentation as syntax (off-side rule)?
hasMacros	Does the language have a compile-time macro pass?
hasPatternMatching	Does the language support pattern matching?
hasPointers	Does the language have pointer types?
hasMixins	Does the language allow mixin-style multiple inclusion?
isCaseSensitive	Is the language case-sensitive?
canDoShebang	Does the language support shebang (#!) scripts?
githubCopilotOptimized	Is the language listed as Copilot-optimized by GitHub?

---

ConceptRelationship

Not a separate file. These are fields stored directly on the source Concept's .scroll file as space-separated lists of target Concept IDs. In pldb.json they appear as space-separated strings (e.g. "influencedBy": "c lisp algol").

This entity represents the directed graph edges between Concepts. The 15 relationship types are:

Relationship	Meaning
influencedBy	What languages influenced the design of this one?
dialectOf	Is this a dialect of a broader language?
supersetOf / subsetOf	Is this a strict superset/subset of another? (Note: a supersetOf link causes the superset to inherit all features of the subset in computed metrics.)
implementationOf	Is this an implementation of another language spec?
forkOf	Was this language forked from another?
extensionOf	Is this an extension of another language?
successorOf	Was this launched as the official successor to another?
renamedTo	What did this language get renamed to?
writtenIn	What language(s) is the main implementation written in?
compilesTo	What language(s) does this primarily compile to?
runsOnVm	What VM(s) does this language run on?
inputLanguages	What language(s) does this tool take as input?
protocols	What protocols does this speak?
related	Catch-all: loosely related concepts (use a more specific type if possible)

All relationships are many-to-many. The inboundLinksCount computed field on Concept counts how many other concepts point to it across all relationship types.

---

GithubRepo

Source: githubRepo sub-tree in concepts/<id>.scroll
In pldb.json: flattened as githubRepo_stars, githubRepo_forks, etc.
Coverage: ~1,179 concepts

The URL and stats for the language's official GitHub repository (e.g. https://github.com/python/cpython). Populated only for languages that are hosted on GitHub. Not to be confused with GithubLanguage (which is about GitHub's Linguist classification) or GithubBigQuery (which is about aggregate usage statistics across all repos).

Field	Notes
url	GitHub repo URL
stars, forks, subscribers, issues	Snapshot counts
created	Year the GitHub repo was created
updated	Year of the most recent commit
description	GitHub repo description

---

GithubLanguage

Source: githubLanguage sub-tree in concepts/<id>.scroll
In pldb.json: flattened as githubLanguage_repos, githubLanguage_type, etc.
Coverage: ~451 concepts

Data from GitHub's Linguist library, which GitHub uses to detect languages in repositories and render syntax highlighting. A language having a GithubLanguage record means GitHub formally recognises it and has data on how many repos use it.

Field	Notes
id	Language name as Linguist knows it
repos	Number of GitHub repos using this language
type	programming, data, markup, prose, or nil
aliases	Alternative names Linguist recognizes
interpreters	Associated interpreter executable names
aceMode, codemirrorMode, tmScope	Editor/highlighting identifiers
trendingProjects	PSV blob of trending repos (crawled snapshot)
group	Parent language group name (e.g. C for C++)
fileExtensions, filenames	Extensions and filenames Linguist associates with this language

---

GithubBigQuery

Source: githubBigQuery sub-tree in concepts/<id>.scroll
In pldb.json: flattened as githubBigQuery_users, githubBigQuery_repos
Coverage: ~420 concepts

Aggregate usage statistics from Google's BigQuery public dataset of all public GitHub repositories. This is a one-time crawled snapshot (not live), giving a broader measure of language prevalence across all of GitHub rather than just trending repos.

Field	Notes
id	Language name in the BigQuery dataset
users	Unique GitHub users with at least one repo in this language
repos	Total repos in this language according to BigQuery snapshot

---

Wikipedia

Source: wikipedia sub-tree in concepts/<id>.scroll
In pldb.json: flattened as wikipedia_summary, wikipedia_appeared, etc.
Coverage: ~1,974 concepts

Data extracted from the concept's Wikipedia page, if one exists. This is one of the most widely populated value objects (covering ~38% of all concepts). The related field is notable: it contains a space-separated list of other Concept IDs that Wikipedia lists as related languages, creating an additional cross-concept link.

Field	Notes
url	Full Wikipedia page URL
summary	First paragraph(s) of the Wikipedia article
appeared	Year Wikipedia claims the language first appeared
created	Year the Wikipedia page was created
dailyPageViews	Average daily page views (used as a popularity signal in ranking)
backlinksCount	Number of Wikipedia pages linking to this page
revisionCount	Number of edits/revisions to the Wikipedia page
pageId	Wikipedia's internal integer page ID
related	Space-separated Concept IDs that Wikipedia lists as related

---

RepoStats

Source: repoStats sub-tree in concepts/<id>.scroll
In pldb.json: flattened as repoStats_commits, repoStats_linesOfCode, etc.
Coverage: ~1,235 concepts

Low-level git statistics for the language's primary source repository (whichever of githubRepo, gitlabRepo, sourcehutRepo, or gitRepo is the main one). These stats are computed by cloning or inspecting the repo directly, not via a GitHub API.

Field	Notes
firstCommit	Year of the earliest commit in the repo
newestCommit	Year of the most recent commit
commits	Total commit count
committers	Number of distinct committers
files	File count in the repo
mb	Size of .git/objects in megabytes
linesOfCode	Total lines of code (computed by cloc)

---

Tiobe

Source: tiobe sub-tree in concepts/<id>.scroll
In pldb.json: flattened as tiobe_currentRank
Coverage: ~204 concepts

The language's presence and ranking on the TIOBE Index, a widely cited monthly ranking of programming language popularity based on web search engine results.

Field	Notes
id	Language name as TIOBE knows it
currentRank	Most recently recorded TIOBE rank (lower = more popular)

---

StackOverflowSurvey

Source: stackOverflowSurvey sub-tree in concepts/<id>.scroll
Coverage: 0 records currently populated in pldb.json

Defined in the schema (measures.parsers) as a yearly map of data from the Stack Overflow Annual Developer Survey. The sub-document is keyed by year, so each concept can have one entry per survey year. Despite being in the schema, it is not populated in the current build — making it a placeholder for future data ingestion.

Field	Notes
year	Survey year (key)
users	Number of developers reporting use of this language
medianSalary	Median annual salary reported by users of this language
fans	Number of developers wanting to learn this language
percentageUsing	Fraction of all survey respondents using this language

---

Subreddit

Source: subreddit sub-tree in concepts/<id>.scroll
In pldb.json: flattened as subreddit_memberCount
Coverage: ~115 concepts

The official (or most popular unofficial) Reddit community for a language. Member counts are stored as a map<year, int> so growth over time can be tracked.

Field	Notes
url	Full subreddit URL (e.g. https://reddit.com/r/Python)
memberCount	Annual snapshots of subscriber count, keyed by year

---

Meetup

Source: meetup sub-tree in concepts/<id>.scroll
In pldb.json: flattened as meetup_memberCount, meetup_groupCount
Coverage: ~58 concepts

Data from Meetup.com on active user groups organized around a language. Gives a signal for community size in the physical/IRL sense.

Field	Notes
url	URL to the Meetup.com topic or group page
memberCount	Total members across all groups for this language
groupCount	Number of distinct Meetup groups

---

DomainName

Source: domainName sub-tree in concepts/<id>.scroll
In pldb.json: flattened as domainName_registered
Coverage: ~889 concepts

Information about the language's own dedicated domain name, if it has one (e.g. python.org, rust-lang.org). The AWIS rank is a historical Alexa Web Information Service domain ranking, stored as an annual map.

Field	Notes
name	Domain name (e.g. python.org)
registered	Year the domain was first registered
awisRank	Annual snapshots of the Alexa domain rank, keyed by year

---

LinkedInSkill

Source: linkedInSkill sub-tree in concepts/<id>.scroll
Coverage: ~57 concepts

A snapshot of how many LinkedIn users list a language as a skill on their profile. Stored as an annual map to capture trends over time. Sparsely populated — only covers languages prominent enough to appear in LinkedIn's skills taxonomy.

Field	Notes
id	Skill name as LinkedIn knows it
annualCount	Snapshot of profile count per year

---

IndeedJobs

Source: indeedJobs sub-tree in concepts/<id>.scroll
Coverage: ~111 concepts

A snapshot of how many job postings on Indeed.com match a search query for the language. Stored as an annual map. Used to estimate numberOfJobsEstimate on Concept.

Field	Notes
query	The search query used on Indeed (often just the language name)
annualCount	Snapshot of matching job listing count per year

---

ProjectEuler

Source: projectEuler sub-tree in concepts/<id>.scroll
Coverage: ~104 concepts

Data from Project Euler, a site hosting mathematical programming problems. The member count shows how many users solved problems using a given language, serving as a niche but precise signal of language adoption among a mathematically-oriented programmer audience.

Field	Notes
id	Language name as Project Euler knows it
memberCount	Annual snapshots of solver count per language, keyed by year

---

LinguistGrammarRepo

Source: linguistGrammarRepo sub-tree in concepts/<id>.scroll
Coverage: ~416 concepts

The GitHub repository containing the TextMate/Linguist grammar used by GitHub to syntax-highlight this language. A language having a LinguistGrammarRepo is a reliable signal it has at least 200 unique :user/:repo repos on GitHub (per Linguist's inclusion criteria).

Field	Notes
url	URL to the grammar repo on GitHub
commitCount	Total commits in the grammar repo
firstCommit	Year of the earliest commit
lastCommit	Year of the most recent commit
sampleCount	Number of language sample files in the repo
committerCount	Number of people who have committed to the repo

---

PygmentsHighlighter

Source: pygmentsHighlighter sub-tree in concepts/<id>.scroll
Coverage: ~341 concepts

Entry in the Pygments syntax highlighting library, which is widely used across documentation systems (Sphinx, GitLab, many static-site generators). A language having a Pygments highlighter is another indicator of meaningful real-world adoption.

Field	Notes
id	Language name in Pygments
filename	Python source filename implementing the lexer (e.g. python.py)
fileExtensions	File extensions Pygments associates with this language

---

CrawledBlobs

Source: Several sub-trees in concepts/<id>.scroll
In pldb.json: stored as large string values
Coverage varies: semanticScholar ~315, isbndb ~390, goodreads ~200, hackerNewsDiscussions ~180, dblp ~0

A grouping of five sub-documents that are scraped/crawled from external databases and stored as raw pipe-separated-value (PSV) blobs inside the Concept record. Unlike the other value objects these are not parsed into typed fields — they are treated as opaque text dumps intended for display or downstream processing.

Sub-document	Source	What it contains
semanticScholar	Semantic Scholar	Academic papers about the language
goodreads	Goodreads	Books about the language listed on Goodreads
isbndb	ISBNdb	Books about the language from the ISBNdb catalogue
hackerNewsDiscussions	Hacker News	HN posts/discussions about the language
dblp	DBLP	Computer science bibliography entries — currently 0 populated

Concept.bookCount is computed from goodreads + isbndb record counts.
Concept.paperCount is computed from semanticScholar record counts.