Distributed HDBSCAN

[graceli24]

• Add implementation of distributed HDBSCAN, found in src/clustering/run_distributed_hdbscan_clustering.cpp and helper files.
• Update script/benchmark/run_clams_bench.py to accept a flag --distributed_hdbscan to run the distributed implementation instead of the serial implementation.
• In both the serial and distributed HDBSCAN implementations, add when the edge distance is 0, calculate lambda = 1/distance to be the max possible value of distance_t. This is in an effort to reduce undefined inf or nan behavior if there are duplicate points with distance 0 between them.
• Add a note in the readme that its best to deduplicate data first before running the code.

[graceli24]

There is currently an inconsistent bug that sometimes the distributed HDBSCAN code gets stuck using YGM multi_output to write cluster labels or data to file. It seems to reach that part of the code and then get hung up on creating the new directory to dump the files to. I think this is inconsistent and sometimes the writing to file works smoothly and other times it gets stuck.

[KIwabuchi]

@graceli24 Looks good to me! Feel free to merge it when it's convenient for you.