SSO/SAML support

[pimguilherme]

Hi dear stackstorm maintainers/community!

I've been working on a PR to implement SSO/SAML using whatever we currently have as a base (st2-auth-backend-sso-saml2) and some other bits and pieces of SSO through the main st2 repository.

I asked around in slack if this had any traction and apparently not, so I just went ahead and started implementing some of it because we need that for our strategy at the company I work for.

Are there any other guidelines regarding contribution other than these items?
https://github.com/StackStorm/st2/blob/abb694b85f75dc543ecea7df6b87def4bce53309/CONTRIBUTING.rst
https://docs.stackstorm.com/development/index.html

I am currently focusing on getting it working on manual tests and will write proper tests afterwards, but can I count on some assistance to further validate/refine this as it progresses?

here's a bit of how it looks so far..
https://user-images.githubusercontent.com/1406885/164324464-29150514-1b5c-415d-bb71-d1e5f2cf568a.mp4

if you have any SAML/SSO material, that would be lovely as well.. currently I've been trying to base the implementation off gravitational/teleport

thanks!!

[rush-skills]

@pimguilherme This demo part looks great. I wanted to test out SSO login, but couldn't find any proper implementation method, however this looks like a good start.

A few questions:

1. Do you handle any groups/roles level permissions while authenticating with SSO?
2. Does it work in parallel to ldap-auth or replaces it?
3. How does the CLI login handled in this case?

[pimguilherme]

thanks for the input @rush-skills !

1 - not yet... i have not looked into the RBAC handling on stackstorm, but if it is as simple as I think it might be, it would just be a matter of mapping groups to roles from the SSO backend (so just a config in the backend should do it)

2 - it should work in parallel to ldap but I am not sure.. the SSO backend as it's currently implemented is parallel to the standard auth mechanisms
3 - the CLI login is somewhat like this:

• user issues 'st2 login --sso'
• the CLI creates a local HTTP (unencrypted) web server on the user PC, at a random port, with urls:
  • /*********** -> random string to initiate the login flow on the browser
  • /callback -> endpoint for the browser to finish the login flow with the SSO response
  • /success -> redirect endpoint to just show the login succesful details
• the CLI creates a random AES key
• the CLI issues a SSO request on the auth componnt at /auth/sso/request/cli, providing the random AES key and the callback URL. the key will be used to encrypt the response back to the CLI
  • the auth generates a random ID for this SSO request, and creates a SSO request object in the database (containing the ID, expiration time (eg 2min), the key, and the type (eg web/cli))
  • the auth component then sends the request to the SSO backend, providing the ID it should use for the request (the same as the SSO request above)
  • the SSO backend does its thing and provides back the SSO URL, that should be used for the user to login on the browser..
  • as part of the login flow, it's configured to send the SSO response back to ST2 at /auth/sso/callback
  • the SSO URL is propagated back to the CLI on the response to this SSO request
• the CLI then provides the user with the message "Please finish the SSO login in the URL:".. providing the local web server's URL
  • once this is done, the CLI sits idle waiting for the response at /callback, for 90 seconds, or it times out
• the user visits the local URL (e.g. http://localhost:42343/ee4b73ff-73df-4f3c-9830-026f913bc072), and then gets redirected to the proper IDP
  • the IDP does the authentication, and then redirects the user to /auth/sso/callback, with the SSO response
  • the auth component parses the incoming response, and asks the SSO backend for the ID associated with that response
  • the auth backend then validates if the ID is valid by checking the database, and whether it has expired.. upon validation, the request is deleted from the DB and may not be used again
  • once the SSO request itself is validated, the auth component goes back to the SSO backend and asks it to parse it and provide the associated details of the logged in user
    • here we would (i guess) just need to do a role/group mapping and provide back to the SSO backend for rbac to work
  • the auth component gets these details and then creates a token
  • the auth backend detects that this was a CLI request (which needs key encryption due to the HTTP webserver locally), and then encrypts the content with the key
  • once encrypted, the content is sent back to the CLI using the callback url (e.g. http://localhost:42343/callback), and the user is redirected to /success for the OK message on the browser
• the CLI decrypts the content with the key and stores the token just like it used to do with regular auth
• done :)

this is more or less what goes on.. in the CLI case there's this encryption key, but in the web case I think we just need to do some cross-site request checks, but that's still to be implemented and validated

[rush-skills]

@pimguilherme These steps seem good to me.

Can you open your PR(s) and I can review the code/test the feature and come back at you better

[pimguilherme]

Hi @rush-skills ! Thanks for getting back to me :)

I will create the PR in the code this week and let you know! Cheers

[rush-skills]

Hey @pimguilherme
Were you able to make any progress on this? Something you need help with?

[pimguilherme]

Hey @rush-skills ! Thanks for reaching out.. unfortunately I did not make any progress as there were other internal priorities in the company.. many of them involved in internal stackstorm automations but this PR got a bit of a halt.. gladly we should be putting more effort into it starting this week.. hopefully to resolve and finish it :)

Here are our prs:

• st2web - Feat/saml st2web#983
• saml_backend - Implementing SAML integration with st2 st2-auth-backend-sso-saml2#16
• st2 -> Implementing SSO/SAML login :) #5664

Some next items to be followed up on:

• finish the SSO/SAML web authentication and security
• figure out which configuration is useful in the config file
• finish RBAC details
• finish the tests

Cheers!

[rush-skills]

Hey @pimguilherme

That looks great and a lot of effort. Thanks for the contribution!

If you can start adding more docs and descriptions to the PR, I can try setting them up next week and testing the changes.

[pimguilherme]

Thanks for the feedback @rush-skills ! It's still not fully done as automated tests and some tweaks here and there are still needed, but it can definitely be tested manually as as starting point.. I am finally going to work on in again this week, so I will provide some more updates and all

cheers :)

[pimguilherme]

hey @rush-skills , I am starting by trying to make tests here:
StackStorm/st2-auth-backend-sso-saml2#16

are you familiar with this code? it looks like the makefile is broken and it doesn't look like other st2-auth-backend modules use a Makefile in general.. I just wanted to check before doing any changes, otherwise I will just try to tinker around and make it work

cheers!

[rush-skills]

hey @rush-skills , I am starting by trying to make tests here: StackStorm/st2-auth-backend-sso-saml2#16

are you familiar with this code? it looks like the makefile is broken and it doesn't look like other st2-auth-backend modules use a Makefile in general.. I just wanted to check before doing any changes, otherwise I will just try to tinker around and make it work

cheers!

Haven't got much experience, but indeed the tests are broken for the repo. From what I see, it still uses CircleCI (we are migrating to GitHub Actions now) with Python 2 build (which again is deprecated). I can try creating an issue and picking it up later, but that won't be soon I guess.

Feel free to tinker around and change the tests however you want in the meanwhile.

[pimguilherme]

hi @rush-skills ! I have deleted some old comments and compiled the latest status:

To implement this change I had to update the following repositories:

• st2 (Implementing SSO/SAML login :) #5664)

  • everything basically ready

  • code changes

  • st2auth + st2common

  • added saml-backend as a default dependency of st2auth (still need to point to st2's org, it's currently using my own repo)

  • added cli/web login endpoints on SSO request (GET /auth/sso/request/web or POST /cli)

  • added SSO request database model for storing pending/requested logins and validate callbacks and expiration

  • added a UUID request_id to SSO requests

  • added a BaseSingleSignOnBackendResponse class to better communicate response with sso backends

  • added remote group + role syncing to the ProxyAuthHandler, based on provided proxy groups on the request (reusing same code as the SStandaloneAuthHandler

  • added SSO backend required method get_request_id_from_response, to be able to check agaisnt pending SSO requests in the database

  • created a TokenResourceManager to implement some SSO logic in to token generation

  • updated openapi endpoints for sso

  • addede some helpers methods to st2common access services

  • added a crypto method to read keys from dict directly

  • st2client

  • added --sso option to CLI and integrated with the backend (still needs some tests/validation but should be working)

  • copied crypto.py code to the st2client package for key encryption (not sure how to deal with this best)

  • added a SSOInterceptorProxy class responsible for spinning up a local server to wait for incoming callback from the backend, with the auth details :)

  • misc details

  • prior to processing callbacks, the incoming SSO request id is validated and database model is deleted so it's not longer usable

  • sso uses ProxyAuthenticationHandler

  • callback response differs between CLI and WEB initial requests

    • cli-based login has encrypted response based on incoming key provided by caller (since the callback to the CLI webserver is usually HTTP)
    • web-based login callback populates the st2session cookie so the user is authenticated and redirects them to normal usage of the GUI

• st2web (https://github.com/StackStorm/st2web/pull/983/files)

  • should be 100% ready (hopefully haha)
  • added a login button that redirects to /auth/sso/request/web
  • configurable via config.json
  • added some tests

• st2-auth-backend-sso-saml2 (Implementing SAML integration with st2 st2-auth-backend-sso-saml2#16)

  • updated some pylint definitions from base repository
  • updating pysaml2 lib to latest
  • implemented and tested the saml2 backend and integration with the SSO handler
  • added some fixtures for testing with keycloak responses
  • added some logging :)

• st2-dockerfiles (adding xmlsec1 to auth dependenci st2-dockerfiles#55)

  • adding xmlsec1 as a dependency to st2auth container

• st2-docker (https://github.com/StackStorm/st2-docker/pull/246/files)

  • added a local keycloak at 3011 with pre-populated stanley user and StackStorm realm
  • enabling RBAC by default
  • enabling SSO by default (in the backend and gui)
  • adding RBAC mappings to st2admin (via assignments) and stanley (via group mapping)

• st2docs (Adding SSO/SAML2 details to the docs st2docs#1146)

  • added SAML2 related documentation (both SSO config + RBAC config)

to test this, you should:

mkdir /tmp/st2-test
cd /tmp/st2-test
# clone test docker files
git clone git@github.com:pimguilherme/st2-dockerfiles.git
cd st2-dockerfiles

# checkout feat/saml-test
git checkout feat/saml-test


#build images base and then st2auth and st2web locally
docker build -t stackstorm/st2:3.8dev base --build-arg ST2_VERSION=3.8dev
docker build -t stackstorm/st2auth:3.8dev st2auth --build-arg ST2_VERSION=3.8dev
docker build -t stackstorm/st2web:3.8dev st2web --build-arg ST2_VERSION=3.8dev

cd /tmp/st2-test

# clone st2-docker modified with keycloak
git clone https://github.com/pimguilherme/st2-docker
cd st2-docker

git checkout feat/saml-backend

# cleanup before :)
docker-compose down

# cleanup old volumes (BEWARE IF YOU DONT WANT TO DELETE LOCAL TEST DATA)
docker volume rm $(docker volume ls | grep -Po '(docker_stackstorm|st2)-\S+')

# bring up the composer
ST2_VERSION=3.8dev docker-compose up -d

# wait for st2web to be healthy (you may need to restart it once)
# login to web ui at http://localhost/
# click the SSO button
# login user with `stanley/Ch@ngeMe`

to test the st2cli:

cd /tmp/st2-test
git clone https://github.com/pimguilherme/st2
cd st2
git checkout feat/saml
make .st2client-install-check
. ./virtualenv-st2client/bin/activate

# then you just run st2 normally :) (-s = --sso)
st2 login -s 

i think this is pretty much it!! please let me know

cheers!

[rush-skills]

@pimguilherme Wow, this is a lot of work. Thanks for the update!

I haven't got the chance to test these changes yet, will try to follow the instructions and test them next week.

Cheers,
Ankur