Upload fails with RemoteDisconnected when a chef takes a long time to build #883

❌ This issue is not open for contribution. Visit Contributing guidelines to learn about the contributing process and how to find suitable issues.

Target branch: main

Observed behavior

After a long chef run, the upload fails on the first request ricecooker makes to Studio, POST /api/internal/create_channel, with RemoteDisconnected. The run dies right after the Creating channel <title> log line, before any nodes are uploaded.

The size of the channel doesn't matter to Studio for this request. What matters is how long the run takes: config.SESSION connects to Studio at login, then sits idle while the channel is built and its files are downloaded. If that takes more than around 5 to 10 minutes, the connection has been dropped by the time create_channel reuses it. urllib3 doesn't detect the drop and doesn't retry the POST. A smaller build of the same chef succeeds.

Reported on the forum: https://community.learningequality.org/t/sushichef-script-to-import-a-channel-failing-with-connection-aborted/4073

Errors and logs

From the forum report (the end of the user's log):

INFO    Uploading files...
INFO      Uploading 0 new file(s) to Kolibri Studio...
INFO    Creating channel...
INFO    Creating tree on Kolibri Studio...
INFO      Creating channel How to Think Like a Computer Scientist, Interactive Edition

urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

The chef in Steps to reproduce fails with the same output.

Expected behavior

Uploading works however long the build took.

User-facing consequences

Chefs with large channels can't upload at all, and the error doesn't explain why. The person running the chef ends up retrying the full run, trimming the channel, or creating new channels to work around it. The only workaround is --resume, which skips the build so the upload starts soon after login.

Steps to reproduce

1. Save this chef as sushichef.py. The sleep stands in for a large channel that takes a while to build:

   import time
   
   from le_utils.constants import exercises
   from le_utils.constants import licenses
   from ricecooker.chefs import SushiChef
   from ricecooker.classes.nodes import ExerciseNode
   from ricecooker.classes.questions import SingleSelectQuestion
   
   
   class IdleConnectionChef(SushiChef):
       channel_info = {
           "CHANNEL_SOURCE_DOMAIN": "example.org",
           "CHANNEL_SOURCE_ID": "ricecooker-idle-connection-repro",
           "CHANNEL_TITLE": "Idle connection repro",
           "CHANNEL_LANGUAGE": "en",
       }
   
       def construct_channel(self, **kwargs):
           channel = self.get_channel(**kwargs)
           channel.add_child(
               ExerciseNode(
                   source_id="exercise",
                   title="Exercise",
                   license=licenses.CC_BY,
                   copyright_holder="Learning Equality",
                   exercise_data={"mastery_model": exercises.M_OF_N, "m": 1, "n": 1},
                   questions=[
                       SingleSelectQuestion(
                           id="q1",
                           question="1 + 1 = ?",
                           correct_answer="2",
                           all_answers=["1", "2", "3"],
                       )
                   ],
               )
           )
           # Stand-in for a large channel that takes a while to build
           time.sleep(11 * 60)
           return channel
   
   
   if __name__ == "__main__":
       IdleConnectionChef().main()

2. Run python sushichef.py --token=<studio token>.
3. After about 11 minutes, the run fails right after Creating channel Idle connection repro with the RemoteDisconnected error.

Context

• ricecooker v0.8.0 (main has the same code path)
• Studio: production (api.studio.learningequality.org, ricecooker's default)
• Reproduced on Ubuntu (forum report) and macOS, Python 3.11

AI usage

I used Claude Code to trace the failure, reproduce it, and draft this issue. I reviewed and edited it.

Acceptance Criteria

• The chef in Steps to reproduce uploads successfully, even when the time between login and "Creating channel" is longer than the connection idle limit.
• Nothing is sent to Studio twice, so no channel or node ends up duplicated.
• An automated test covers a run where the connection to Studio is dropped between login and upload.