Support for Dictionary Observation Space

[damaruga]

Problem

Robotic Env such as SurRoL and Fetch uses Dictionary Observation Space, with

1. observation
2. desired_goal
3. achieved_goal
   as keys.

Query

Is there a quick fix to train an agent using BC in the current release for these kinds of environments?

[ernestum]

It is not supported right now:
#651 (comment)

We are currently reworking the way data is handled in general. @AdamGleave do you think there is a quick fix? Should we aim to support this in the long term?

Maybe you can use the space utils to flatten/unflatten the space?
https://gymnasium.farama.org/api/spaces/utils/

[tudorjnu]

Hello! I have a similar issue as I need to use a dictionary input method. If one was to implement this, where would be a good starting point?

[ernestum]

As long as you don't plan on treating different parts of the observation space (e.g. applying a CNN to image data but taking in accelerator data directly) you should be good with a gym.wrappers.FlattenObservation wrapper around your environment.

[tudorjnu]

Personally that is roughly what I need as I have a custom environment generating different types of observations (i.e. image, joint pos, etc.).

On a tangent note, is it possible to generate transtitions using an environment (i.e. dict based env) and store only the image for example and then use imitation to learn those transitions (image -> action)?

I have been thinking to create a custom rollout wrapper or something along those lines so that when the expert generates the trajectories, they can be filtered. What do you think?

I asked here as it seems to be related to the issue. Thank you!

[NixGD]

PR #785 partially addresses this. Once it's merged dictionary observations should be supported in:

Core functionality

• Collecting rollouts
• Saving / writing trajectories to disk
• Buffers

Algorithms:

• Behavioral Cloning
• DAgger
• Density based reward modelling
• MCEIRL
• Adversarial AIRL / GAIL
• SQIL
• Preference Comparisons

[benediktkreis]

I've seen that the PR #785 was merged. Are there any updates regarding the other algorithms? In particular, I'm interested in dict support for DAgger.

[benediktkreis]

Are there any updates regarding dictionary support for DAgger?