Issue #4193 : Define lifecycle environments in the project configuration

docs/hop-user-manual/modules/ROOT/pages/projects/projects-environments.adoc

@@ -36,7 +36,7 @@ image:hop-gui/environment/add-project-menu.png[Add project menu]
 
 *From version control* checks out a project from a Git repository instead.
 
-The project properties fields are grouped on four tabs: *Basic*, *Folders*, *Parent project*, and *Variables*.
+The project properties fields are grouped on five tabs: *Basic*, *Folders*, *Parent project*, *Variables*, and *Environments*.
 
 image:hop-gui/environment/create-project-dialog.png[Project Properties Basic tab,width="90%"]
 
@@ -107,7 +107,57 @@ image:hop-gui/environment/create-project-dialog-variables-tab.png[Project Proper
 |Project variables to set|A list of variable names, values and variable descriptions to use with this project|No|No|
 |===
 
-After creating a project the user interface will switch to it and ask if you want to create an environment.
+== Embedded environments
+
+The *Environments* tab stores zero, one, or more lifecycle environment definitions in `project-config.json`.
+These definitions are meant to be checked in with the project.
+Each one has a name, a description, and a list of variables.
+Each variable has a *Mandatory* flag and a *Secret* flag.
+A mandatory variable is one a person has to set on their own computer.
+A secret is a password, token, or similar value.
+Both flags can be set on the same variable.
+The definition does not contain configuration files.
+
+The *Default value* of a variable is checked in with the project.
+A value that is the same on every computer, such as a log level or a host name, can be stored as it is.
+Use a placeholder such as `change-to-your-password` when the real value should stay on one computer.
+Clear a real secret before you save when it should stay on this computer.
+
+*Add*, *Edit...*, *Copy...*, *Delete*, and *Import...* change the list.
+*Edit...* sets the name, the description, and the variables.
+*Copy...* asks for a new name and copies the description and the variables.
+*Import...* reads one or more lifecycle environments of this project from the Hop configuration on this computer.
+For each variable in those configuration files you set *Secret* and *Mandatory*.
+The value from the configuration file is copied in as the *Default value*, including a secret.
+You can change or clear that default before it is saved in the project.
+When the same name appears in more than one configuration file, the last value that is not blank is kept.
+A blank value does not clear an earlier one.
+The first description that is not blank is kept.
+
+*Implement* creates the lifecycle environments on this computer from the definitions in the list.
+It asks the same question as when a project is added.
+The project file is saved when you click OK.
+
+When a project is added from a folder or from Git and `project-config.json` already defines environments, Hop asks whether to create those lifecycle environments on this computer.
+A project with no embedded environments still asks the generic question about creating one environment, except when the project is read only or was added from Git.
+
+Every variable marked *Secret* or *Mandatory* is written, one configuration file per environment.
+A variable that is neither is applied when the environment is enabled and is not stored in that file.
+A secret is written whether or not it is mandatory.
+The value written for each of those variables is its default from the project, including a placeholder such as `change-to-your-password`.
+Replace it on this computer when that default is only a placeholder.
+A value in the configuration file replaces the default.
+A variable with the same name on the project still wins.
+
+The files are written to the folder in the project variable `+${ENVIRONMENTS_FOLDER}+` when that variable is set.
+Otherwise Hop asks for a folder.
+Keep that folder outside the project so the values are not checked in.
+A typical place is `+${HOP_CONFIG_FOLDER}/environments/+` followed by the project name.
+Set `+${ENVIRONMENTS_FOLDER}+` on the project to reuse the same folder the next time the project is added.
+An environment that already exists on this computer is left unchanged.
+An existing configuration file is not overwritten. The environment is still registered when the name is free.
+
+In the environment properties dialog, *Embedded environment* links the environment to one of these definitions by name.
 
 == Create an environment
 
@@ -136,6 +186,7 @@ image:hop-gui/environment/environment-dialog-general-tab.png[Environment Propert
 * Common build
 * ...|No|No|
 |Project|The project to which this environment belongs|No|No|The last created project
+|Embedded environment|The environment definition in the project to inherit. Required values and secrets stay in the configuration files. The other variables come from the project definition|No|No|(none)
 |Canvas text|Large text drawn in the top-right of pipeline and workflow canvases when this environment is active|Yes|No|
 |===
 

plugins/misc/projects/src/main/java/org/apache/hop/projects/environment/EmbeddedEnvironment.java

@@ -0,0 +1,150 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *       http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.projects.environment;
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import java.util.ArrayList;
+import java.util.List;
+import lombok.AccessLevel;
+import lombok.Getter;
+import lombok.NoArgsConstructor;
+import lombok.Setter;
+
+/**
+ * Lifecycle environment definition stored in {@code project-config.json}.
+ *
+ * <p>Holds the name, description, and variable defaults that are shared through version control.
+ * Configuration files stay on the computer that runs Hop.
+ */
+@Getter
+@Setter
+@NoArgsConstructor
+public class EmbeddedEnvironment {
+
+  private String name;
+
+  private String description;
+
+  /**
+   * Variables for this environment. A variable with {@code mandatory} or {@code secret} set is
+   * written to the local configuration file. The others are applied as defaults only.
+   */
+  private List<EmbeddedEnvironmentVariable> variables = new ArrayList<>();
+
+  /**
+   * {@code mandatoryVariables} from project files written before mandatory was a flag on each
+   * variable. Folded into {@code variables} by {@link #absorbLegacyVariables()}.
+   */
+  @JsonIgnore
+  @Getter(AccessLevel.NONE)
+  @Setter(AccessLevel.NONE)
+  private List<EmbeddedEnvironmentVariable> legacyMandatoryVariables;
+
+  /**
+   * {@code secretVariables} from project files written before secret was a flag on each variable.
+   * Folded into {@code variables} by {@link #absorbLegacyVariables()}.
+   */
+  @JsonIgnore
+  @Getter(AccessLevel.NONE)
+  @Setter(AccessLevel.NONE)
+  private List<EmbeddedEnvironmentVariable> legacySecretVariables;
+
+  /**
+   * @return a deep copy
+   */
+  public EmbeddedEnvironment copy() {
+    absorbLegacyVariables();
+    EmbeddedEnvironment copy = new EmbeddedEnvironment();
+    copy.name = name;
+    copy.description = description;
+    copy.variables = copyVariables(variables);
+    return copy;
+  }
+
+  /**
+   * Accept a project file that still stores mandatory variables in their own list. Each of those
+   * variables is appended to {@code variables} with {@code mandatory} set.
+   */
+  @JsonProperty(value = "mandatoryVariables", access = JsonProperty.Access.WRITE_ONLY)
+  public void setMandatoryVariables(List<EmbeddedEnvironmentVariable> legacyMandatoryVariables) {
+    this.legacyMandatoryVariables = legacyMandatoryVariables;
+  }
+
+  /**
+   * Accept a project file that still stores secrets in their own list. Each of those variables is
+   * appended to {@code variables} with {@code secret} set.
+   */
+  @JsonProperty(value = "secretVariables", access = JsonProperty.Access.WRITE_ONLY)
+  public void setSecretVariables(List<EmbeddedEnvironmentVariable> legacySecretVariables) {
+    this.legacySecretVariables = legacySecretVariables;
+  }
+
+  /**
+   * Move {@code mandatoryVariables} and {@code secretVariables} into {@code variables}. Safe to
+   * call more than once. Mandatory entries are appended first.
+   */
+  public void absorbLegacyVariables() {
+    absorb(legacyMandatoryVariables, false, true);
+    legacyMandatoryVariables = null;
+    absorb(legacySecretVariables, true, false);
+    legacySecretVariables = null;
+  }
+
+  private void absorb(List<EmbeddedEnvironmentVariable> legacy, boolean secret, boolean mandatory) {
+    if (legacy == null || legacy.isEmpty()) {
+      return;
+    }
+    if (variables == null) {
+      variables = new ArrayList<>();
+    }
+    for (EmbeddedEnvironmentVariable variable : legacy) {
+      if (variable == null) {
+        continue;
+      }
+      if (secret) {
+        variable.setSecret(true);
+      }
+      if (mandatory) {
+        variable.setMandatory(true);
+      }
+      variables.add(variable);
+    }
+  }
+
+  private static List<EmbeddedEnvironmentVariable> copyVariables(
+      List<EmbeddedEnvironmentVariable> source) {
+    List<EmbeddedEnvironmentVariable> copy = new ArrayList<>();
+    if (source == null) {
+      return copy;
+    }
+    for (EmbeddedEnvironmentVariable variable : source) {
+      if (variable == null) {
+        continue;
+      }
+      copy.add(
+          new EmbeddedEnvironmentVariable(
+              variable.getName(),
+              variable.getDefaultValue(),
+              variable.getDescription(),
+              variable.isMandatory(),
+              variable.isSecret()));
+    }
+    return copy;
+  }
+}