ReallyBigSimulations

This repo demonstrates how to use a LocalPreferences.toml (in particular, to use CUDA's unified memory for really big simulations with Oceananigans).

Try it by cloning this package (on a machine that has access to a GPU) and then writing

julia --project this_is_big.jl

which produces

[ReallyBigSimulations]$ julia --project this_is_big.jl
(parent(c)).data = GPUArrays.DataRef{CUDACore.Managed{CUDACore.DeviceMemory}}(GPUArrays.RefCounted{CUDACore.Managed{CUDACore.DeviceMemory}}(CUDACore.Managed{CUDACore.DeviceMemory}(DeviceMemory(216 bytes at 0x0000000340000000), CuStream(0x00000000068c79e0, CuContext(0x0000000007e4f7e0)), true, true, false), CUDACore.pool_free, Base.Threads.Atomic{Int64}(1)), false, false)

notice the CUDACore.DeviceMemory, which indicates that the memory underlying c is device memory.

For example, to get CUDA to allocate unified memory instead, change the LocalPreferences.toml from its current setting,

[CUDACore]
#default_memory = "unified"
default_memory = "device"

to

 [CUDACore] 
 default_memory = "unified" 
 #default_memory = "device"

(You can also delete the line that is commented out.) Re-running our big script then produces

[ReallyBigSimulations]$ julia --project this_is_big.jl
(parent(c)).data = GPUArrays.DataRef{CUDACore.Managed{CUDACore.UnifiedMemory}}(GPUArrays.RefCounted{CUDACore.Managed{CUDACore.UnifiedMemory}}(CUDACore.Managed{CUDACore.UnifiedMemory}(UnifiedMemory(216 bytes at 0x0000400360000000), CuStream(0x000000003f1bdde0, CuContext(0x000000003da24540)), true, true, false), CUDACore.pool_free, Base.Threads.Atomic{Int64}(1)), false, false)

where we can see CUDACore.UnifiedMemory --- as we prefer!