In our project, what we do is we run a shell script which triggers a Python script which in turn triggers our spark application.
Here I am trying to create the same project but not including all the checks and validations and detail level design that we have in our production.