fix(functions): clean up generated artifacts when kit install fails - #11082
fix(functions): clean up generated artifacts when kit install fails#11082wandamora wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces robust error handling and rollback mechanisms during function kit installation, configuration, and instance addition by adding cleanup utilities (safeRemove, cleanUpEmptyDir, and revertFunctionsConfig) and wrapping key operations in try-catch blocks. The review feedback correctly identifies a critical bug where pre-emptively mutating the in-memory configuration in the catch blocks of addInstanceToKit and addKitInstanceOrConfigureProject prevents revertFunctionsConfig from detecting changes and writing the reverted configuration back to firebase.json on disk. The reviewer provides actionable suggestions to remove these pre-emptive deletions and update the corresponding unit test assertions to verify the reverted configuration.
Remove premature in-memory instance deletions in addInstanceToKit and addKitInstanceOrConfigureProject catch blocks so revertFunctionsConfig detects the difference and writes the reverted config back to disk. Update unit tests to assert serialized content on disk rather than object references.
ajperel
left a comment
There was a problem hiding this comment.
I hope I left a suggestion that will make this simpler.
| } | ||
|
|
||
| /** | ||
| * Removes an empty directory if it exists and contains no files or subdirectories. |
There was a problem hiding this comment.
You could also note this suppresses any errors and logging to debug.
| /** | ||
| * Safely removes a file or directory, suppressing any errors and logging to debug. | ||
| */ | ||
| export async function safeRemove(targetPath: string): Promise<void> { |
There was a problem hiding this comment.
This and cleanUpEmptyDir seem fairly generic and might be well served moving to fsutils.ts
| /** | ||
| * Removes an empty directory if it exists and contains no files or subdirectories. | ||
| */ | ||
| export async function cleanUpEmptyDir(absDirPath: string): Promise<void> { |
There was a problem hiding this comment.
maybe an even clearer name would be removeDirectoryIfEmpty
| if (await fs.pathExists(absDirPath)) { | ||
| const stat = await fs.stat(absDirPath); | ||
| if (stat.isDirectory()) { | ||
| const files = await fs.readdir(absDirPath); |
There was a problem hiding this comment.
nit: I'd call this something more generic like entries since it could be a file or a directory.
| */ | ||
| export function revertFunctionsConfig(options: RevertFunctionsConfigOptions): void { | ||
| try { | ||
| const originalFunctionsJson = JSON.stringify(options.originalFunctions); |
There was a problem hiding this comment.
You should use deepEqual from utils.ts probably. Not that it should happen in this case but it will not care about say reordering things and can fail fast.
| /** | ||
| * Reverts the functions configuration in firebase.json to its original state if it was modified. | ||
| */ | ||
| export function revertFunctionsConfig(options: RevertFunctionsConfigOptions): void { |
There was a problem hiding this comment.
This is a cool pattern we might want to expand elsewhere in future PRs and make a shared config thing. But I wouldn't do that in this PR.
| return { configDirPath, absConfigDirPath }; | ||
| } catch (err: unknown) { | ||
| await safeRemove(absConfigDirPath); | ||
| await cleanUpEmptyDir(options.config.path(path.join(FUNCTION_KITS_DIR, options.kitId))); |
There was a problem hiding this comment.
Won't these directories never be empty because you have a source directory already if this is the addInstance case and this is a second instance? I think you just need to clean up configDir and firebase.json
| try { | ||
| await fs.remove(targetPath); | ||
| } catch (err: unknown) { | ||
| logger.debug(`Failed to remove path '${targetPath}': ${getErrMsg(err)}`); |
There was a problem hiding this comment.
This is OK in in our time crunch... but we should think through the UX a bit more. I wonder if on a failure like this we should at the end instead be like:
Warning: Kit install failed. Unable to remove partial install. To clean up this kit/instance:
- Remove X
- Edit Y
| let absSourcePathToPreserve: string | undefined; | ||
|
|
||
| await source.buildAndInstall(absSourcePath); | ||
| try { |
There was a problem hiding this comment.
You have a lot of cleanup logic spread all throughout this file and it's just getting more and more complicated. Why don't we take a simple approach. This file orchestrates everything right?
At the beginning of this you:
- Snapshot firebase.json
- Create an array like
const createdPaths: string[] = [];
We can pass createdPaths in options to all the sub methods (required?) Anytime they create a new file or directory the path is added to createdPaths... one shared object.
Then we have only this try/catch look for cleanup. Any failure we catch here and run:
await Promise.all(this.createdPaths.map(safeRemove));
await cleanUpEmptyDir(this.options.config.path(FUNCTION_KITS_DIR));
revertFunctionsConfig({ config: this.options.config, originalFunctions: this.originalFunctions });
and then can re-throw the error.
There was a problem hiding this comment.
Actually, shouldn't we really even only need to have one path to remove. It'll be either:
- Entire Kit Directory (kit install)
- Entire Config Directory (add instance)
- Single .env file (add Env)
We just need to pipe that one path back here, though maybe still easiest using createdPaths?
Description
When
firebase functions:kits:installfails partway through execution (e.g.npm installfailure, TypeScript compile error, parameter resolution cancellation/error, or build discovery failure), the command previously left behind partially generated artifacts on disk and, in some flows, a modifiedfirebase.jsonreferencing an uninstalled kit or instance. Users had to manually delete files insidefunction-kits/and hand-editfirebase.jsonbefore retrying.This PR makes function kit installation and instance addition transactional. If any step fails during execution, all artifacts created during that invocation are rolled back and
firebase.jsonis restored to its pre-install state before rethrowing the original error.Key Changes:
Cleanup Helpers (
src/functions/kits/install.ts):safeRemove(targetPath): Safely removes files or directories, catching and logging any filesystem errors vialogger.debugso cleanup failures never swallow or mask the root cause.cleanUpEmptyDir(dirPath): Recursively removes directory trees only if they exist and are empty, pruning orphaned parent directories (e.g.,function-kits/<kitId>andfunction-kits/).revertFunctionsConfig({ config, originalFunctions }): Restores in-memoryconfig.src.functionsto a snapshot taken prior to mutation and, if changed, persists the reverted state back tofirebase.json.Transactional Error Handling across Install & Configure Paths:
installKitOrInstance: Scaffolding, dependency installation, build, parameter prompt, and config mutations are wrapped in atry/catch. On failure, removes generated kit/config directories (while preserving user-provided directories when installed via--directory) and revertsfirebase.json.addKitInstanceOrConfigureProject: Wraps adding a new instance or project environment. On failure, removes the created instance config directory or.envfiles and cleans up any empty parent kit directories.addInstanceToKit: Preserves existing config snapshot; rolls back the instance config directory, empty parent directories, andfirebase.jsonon failure.scaffoldKit: Cleans up the target kit directory on unpack or scaffold failure before propagating the error.Comprehensive Unit Testing (
src/functions/kits/install.spec.ts):firebase.jsonand directory structures on failednpm install.--directoryare preserved while CLI-generated config directories are pruned.firebase.jsonduring write calls to ensure configuration rollbacks are written to disk.Scenarios Tested
npm run mocha:fast/npm test.npm install/ build failure): verifiedfunction-kits/<kitId>andfunction-kits/are pruned andfirebase.jsonremains untouched.--directory: verified user source code directory is preserved, generated config directory is deleted, andfirebase.jsonremains untouched.firebase.jsonretains only the original instance..env.<projectId>) failure: verified newly created.envfile is deleted without modifying existing project files.Sample Commands