Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,56 @@ It exposes a well-defined API over a gRPC service that translates all Tsuru oper
[Cloud Build]: https://cloud.google.com/build
[kaniko]: https://github.com/GoogleContainerTools/kaniko

## Remote repository providers

Some container registries (Amazon ECR, Oracle Cloud OCIR) do not create image
repositories on first push — pushes to a nonexistent repository fail (e.g. ECR
returns `404 Not Found`). Since Tsuru names images with one repository per app,
deploy-agent can create the repository before pushing.

Enable it by pointing `--remote-repository-path` (or the
`REMOTE_REPOSITORY_PATH` environment variable) at a JSON file mapping each
registry host to a provider config:

```json
{
"123456789012.dkr.ecr.us-east-1.amazonaws.com": {
"provider": "ecr"
},
"sa-saopaulo-1.ocir.io": {
"provider": "oci",
"compartmentID": "ocid1.compartment.oc1..aaaa...",
"profile": "DEFAULT",
"configPath": "/etc/oci/config"
}
}
```

Registries that auto-create repositories on push (Docker Hub, GCR/Artifact
Registry, Harbor, Distribution) need no entry.

### `ecr` provider

Creates the repository via the AWS API using the default credential chain
(IRSA, instance profile, or environment variables) — the same ambient
credentials used by `docker-credential-ecr-login` for the push itself. The AWS
region is resolved from the optional `"region"` config key, falling back to
the region in the registry hostname, then to the SDK defaults. An
already-existing repository is not an error. The identity needs the
`ecr:CreateRepository` permission on the repository prefix, e.g.:

```json
{
"Effect": "Allow",
"Action": ["ecr:CreateRepository"],
"Resource": "arn:aws:ecr:<region>:<account>:repository/tsuru/*"
}
```

Repositories are created with ECR defaults; to control settings like image
scanning or lifecycle policies, pre-create the repositories with your IaC
tooling instead — the provider treats them as already existing.

## Local Development Setup

To set up your local development environment for deploy-agent, follow these steps:
Expand Down
15 changes: 15 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ toolchain go1.24.9

require (
github.com/alessio/shellescape v1.4.1
github.com/aws/aws-sdk-go-v2/config v1.32.33
github.com/aws/aws-sdk-go-v2/service/ecr v1.60.2
github.com/containerd/console v1.0.3
github.com/docker/cli v23.0.0-rc.1+incompatible
github.com/docker/docker v28.0.0+incompatible
Expand All @@ -29,6 +31,19 @@ require (
cloud.google.com/go/compute/metadata v0.3.0 // indirect
github.com/Microsoft/go-winio v0.6.0 // indirect
github.com/Microsoft/hcsshim v0.9.12 // indirect
github.com/aws/aws-sdk-go-v2 v1.43.2 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.19.32 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.33 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.33 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.33 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.34 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.14 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.33 // indirect
github.com/aws/aws-sdk-go-v2/service/signin v1.5.2 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.33.2 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.2 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.45.2 // indirect
github.com/aws/smithy-go v1.27.5 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/containerd/containerd v1.6.38 // indirect
Expand Down
30 changes: 30 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,36 @@ github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kd
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
github.com/aws/aws-sdk-go v1.15.11/go.mod h1:mFuSZ37Z9YOHbQEwBWztmVzqXrEkub65tZoCYDt7FT0=
github.com/aws/aws-sdk-go-v2 v1.43.2 h1:cl+IXwWb3qazClUcm08tGSsB6OiuV83JVJO9B0jQcPc=
github.com/aws/aws-sdk-go-v2 v1.43.2/go.mod h1:WEzLKBh/mEjXvx1FtQMWgSxMSTVqxQzjkRtk5fa3wkg=
github.com/aws/aws-sdk-go-v2/config v1.32.33 h1:M1m/Q6f0OKDEDGwhiNOqx1OjTdrewe3v+GDbHmKczWk=
github.com/aws/aws-sdk-go-v2/config v1.32.33/go.mod h1:fGj1iQj2QpIZzp7jE4aQQ+71TE8cd4z9K4+xCd6EqmE=
github.com/aws/aws-sdk-go-v2/credentials v1.19.32 h1:eNE0JnIblBo1NCvd3tqEYuZz9XDefn69R74CHd3nT7U=
github.com/aws/aws-sdk-go-v2/credentials v1.19.32/go.mod h1:yYJu+6tqKUYZuJSYcpSGjz/6sV/SUaAaKIufnWKx2OU=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.33 h1:MobhiR6KIerWxmO74Zit5I3379+mSc2DOdZ3DeRFB9w=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.33/go.mod h1:xu02847OdZfNr/jAfZpHtyRk0b3v4d0kaoxNHxZGG/w=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.33 h1:HAp1wLFZzch054uh3FK7rcVYg4v7J2FxVf3h3IGNZas=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.33/go.mod h1:mJk5fmqnF+WUlMdPG37pR2Fh3oh6r8F6ZGUgPKvzu0c=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.33 h1:0YA0aCKgsJyno6xkFfaIgjE3/wK08+Qxo9nQfe1UrWM=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.33/go.mod h1:UZqj4WIdTH+ga8Y/DgpAuy/8cGjM3h7gDCliJYGg2SE=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.34 h1:HQYnjFnXpX8EbPW5M1QT8mXzesRPwly0HEPTcFlS02Y=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.34/go.mod h1:tGzj56niKYZBbDIRhwPGDqrULzmWv5b6uBQGqyNaFZw=
github.com/aws/aws-sdk-go-v2/service/ecr v1.60.2 h1:Lwvln8t0Ll6aXA0p5ZwQcGDIzabi6l+iQ94u80+4bYk=
github.com/aws/aws-sdk-go-v2/service/ecr v1.60.2/go.mod h1:AIS7uTFJRzM4gSxFY7k2fK43mdXwP4D3RVI/nruHsLs=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.14 h1:SA43nfaY7+1jjMNIc2ywu99JLJLButtIdLP6j+bT870=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.14/go.mod h1:Du3llKcwbQvHsTXSLzTOGQz0DTDBMEzdg7DAGu7inrY=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.33 h1:mqI7OrxN/DUH85F5OqVn3cIfuZ3+HVcebUm2N8mLlgQ=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.33/go.mod h1:eZ5jdEpvaaOU8nWWE4cTAJETSEA5FZoWxvNRao4piHY=
github.com/aws/aws-sdk-go-v2/service/signin v1.5.2 h1:EjI1CZzDcBxPkTa3j1BdtIrUDbqnOGssFMeyUS+6W0I=
github.com/aws/aws-sdk-go-v2/service/signin v1.5.2/go.mod h1:vN3eb5H8MEAZ4dx0F5Wc9LT8eb3eW7bZZ5BjGJdbw9k=
github.com/aws/aws-sdk-go-v2/service/sso v1.33.2 h1:zMP1FDFE08L7sM5f1QqkH/ZgKKg8Uc0Dz7KhSSYqWkw=
github.com/aws/aws-sdk-go-v2/service/sso v1.33.2/go.mod h1:0LoIZSUKjdo2BleHfT1hv/jlD33LQS00IrBlzoUsoUQ=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.2 h1:9eTqUYl+SyVmaRPMyBXSO9wwqC6TRwZB82pKENK2hdQ=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.2/go.mod h1:DThweuz22kiLc7lGHop5vQ9c3bx5W6Azs/YqSHa2fu8=
github.com/aws/aws-sdk-go-v2/service/sts v1.45.2 h1:EJd8vZO3E8SE6nmPqxuxlQ1NeSb8as50sf6eGdV4Saw=
github.com/aws/aws-sdk-go-v2/service/sts v1.45.2/go.mod h1:OgpPvKzsO2Ranjpli/20djMkg6UrV5mw4W3pZpq1Mqo=
github.com/aws/smithy-go v1.27.5 h1:d1ro7KpYOYwP6m73YFa+Kc/A130VsAdX68SpsJwARMM=
github.com/aws/smithy-go v1.27.5/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/beorn7/perks v0.0.0-20160804104726-4c0e84591b9a/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
Expand Down
103 changes: 103 additions & 0 deletions pkg/repository/ecr/ecr.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
// Copyright 2026 tsuru authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package ecr

import (
"context"
"errors"
"fmt"
"regexp"
"strings"
"sync"

"github.com/aws/aws-sdk-go-v2/config"
awsecr "github.com/aws/aws-sdk-go-v2/service/ecr"
"github.com/aws/aws-sdk-go-v2/service/ecr/types"
)

// <account-id>.dkr.ecr.<region>.amazonaws.com (with optional FIPS endpoint)
var ecrHostRegexp = regexp.MustCompile(`^\d{12}\.dkr\.ecr(?:-fips)?\.([a-z0-9-]+)\.amazonaws\.com$`)

type ECRRequiredMethods interface {
CreateRepository(ctx context.Context, params *awsecr.CreateRepositoryInput, optFns ...func(*awsecr.Options)) (*awsecr.CreateRepositoryOutput, error)
}

type ECR struct {
client ECRRequiredMethods
Region string
mu sync.Mutex
}

func NewECR(data map[string]string) *ECR {
return &ECR{
Region: data["region"],
}
}

func (r *ECR) Ensure(ctx context.Context, name string) error {
repo, region, err := parseImage(name)
if err != nil {
return err
}
if r.Region != "" {
region = r.Region
}
client, err := r.getClient(ctx, region)
if err != nil {
return err
}
_, err = client.CreateRepository(ctx, &awsecr.CreateRepositoryInput{
RepositoryName: &repo,
})
var alreadyExists *types.RepositoryAlreadyExistsException
if errors.As(err, &alreadyExists) {
return nil
}
if err != nil {
return fmt.Errorf("failed to create ECR repository %q: %w", repo, err)
}
return nil
}

// getClient lazily creates the AWS client; deploys run concurrently, so the
// client field is only accessed under the lock.
func (r *ECR) getClient(ctx context.Context, region string) (ECRRequiredMethods, error) {
r.mu.Lock()
defer r.mu.Unlock()
if r.client != nil {
return r.client, nil
}
var optFns []func(*config.LoadOptions) error
if region != "" {
optFns = append(optFns, config.WithRegion(region))
}
cfg, err := config.LoadDefaultConfig(ctx, optFns...)
if err != nil {
return nil, err
}
r.client = awsecr.NewFromConfig(cfg)
return r.client, nil
}

// parseImage extracts the ECR repository name (the full path after the
// registry host, without tag or digest) and, when the host is a standard ECR
// endpoint, the AWS region embedded in it.
func parseImage(image string) (string, string, error) {
host, repo, found := strings.Cut(image, "/")
if !found || repo == "" {
return "", "", fmt.Errorf("invalid image format %s", image)
}
if i := strings.Index(repo, "@"); i >= 0 {
repo = repo[:i]
}
if i := strings.LastIndex(repo, ":"); i >= 0 {
repo = repo[:i]
}
var region string
if m := ecrHostRegexp.FindStringSubmatch(host); m != nil {
region = m[1]
}
return repo, region, nil
}
125 changes: 125 additions & 0 deletions pkg/repository/ecr/ecr_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
// Copyright 2026 tsuru authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package ecr

import (
"context"
"errors"
"sync"
"testing"

awsecr "github.com/aws/aws-sdk-go-v2/service/ecr"
"github.com/aws/aws-sdk-go-v2/service/ecr/types"
"github.com/stretchr/testify/assert"
)

type FakeECRClient struct {
repos map[string]bool
failWith error
createCalls int
}

func (m *FakeECRClient) CreateRepository(ctx context.Context, request *awsecr.CreateRepositoryInput, optFns ...func(*awsecr.Options)) (*awsecr.CreateRepositoryOutput, error) {
m.createCalls++
if m.failWith != nil {
return nil, m.failWith
}
repo := *request.RepositoryName
if m.repos[repo] {
return nil, &types.RepositoryAlreadyExistsException{}
}
if m.repos == nil {
m.repos = make(map[string]bool)
}
m.repos[repo] = true
return &awsecr.CreateRepositoryOutput{}, nil
}

func TestEnsureCreatesRepository(t *testing.T) {
client := &FakeECRClient{}
r := &ECR{client: client}
err := r.Ensure(context.TODO(), "123456789012.dkr.ecr.us-east-1.amazonaws.com/tsuru/app-hello-world:v1")
assert.NoError(t, err)
assert.True(t, client.repos["tsuru/app-hello-world"])
}

func TestEnsureKeepsFullRepositoryPath(t *testing.T) {
client := &FakeECRClient{}
r := &ECR{client: client}
err := r.Ensure(context.TODO(), "123456789012.dkr.ecr.us-east-1.amazonaws.com/org/sub/tsuru/app-hello-world:v1")
assert.NoError(t, err)
assert.True(t, client.repos["org/sub/tsuru/app-hello-world"])
}

func TestEnsureRepositoryAlreadyExists(t *testing.T) {
client := &FakeECRClient{repos: map[string]bool{"tsuru/app-hello-world": true}}
r := &ECR{client: client}
err := r.Ensure(context.TODO(), "123456789012.dkr.ecr.us-east-1.amazonaws.com/tsuru/app-hello-world:v2")
assert.NoError(t, err)
assert.Equal(t, 1, client.createCalls)
}

func TestEnsureCreateRepositoryError(t *testing.T) {
client := &FakeECRClient{failWith: errors.New("AccessDeniedException: not authorized to perform: ecr:CreateRepository")}
r := &ECR{client: client}
err := r.Ensure(context.TODO(), "123456789012.dkr.ecr.us-east-1.amazonaws.com/tsuru/app-hello-world:v1")
assert.Error(t, err)
assert.Contains(t, err.Error(), "tsuru/app-hello-world")
assert.Contains(t, err.Error(), "AccessDeniedException")
}

func TestEnsureInvalidImage(t *testing.T) {
client := &FakeECRClient{}
r := &ECR{client: client}
err := r.Ensure(context.TODO(), "app-hello-world")
assert.Error(t, err)
assert.Equal(t, 0, client.createCalls)
}

func TestGetClientConcurrent(t *testing.T) {
r := NewECR(map[string]string{"region": "us-east-1"})
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
_, err := r.getClient(context.TODO(), "us-east-1")
assert.NoError(t, err)
}()
}
wg.Wait()
}

func TestNewECR(t *testing.T) {
r := NewECR(map[string]string{"region": "sa-east-1"})
assert.Equal(t, "sa-east-1", r.Region)
}

func TestParseImage(t *testing.T) {
tests := []struct {
image string
repo string
region string
err bool
}{
{image: "123456789012.dkr.ecr.us-east-1.amazonaws.com/tsuru/app-x:v1", repo: "tsuru/app-x", region: "us-east-1"},
{image: "123456789012.dkr.ecr.sa-east-1.amazonaws.com/tsuru/app-x", repo: "tsuru/app-x", region: "sa-east-1"},
{image: "123456789012.dkr.ecr-fips.us-gov-west-1.amazonaws.com/tsuru/app-x:latest", repo: "tsuru/app-x", region: "us-gov-west-1"},
{image: "123456789012.dkr.ecr.us-east-1.amazonaws.com/org/sub/tsuru/app-x:v10", repo: "org/sub/tsuru/app-x", region: "us-east-1"},
{image: "123456789012.dkr.ecr.us-east-1.amazonaws.com/tsuru/app-x@sha256:0000000000000000000000000000000000000000000000000000000000000000", repo: "tsuru/app-x", region: "us-east-1"},
{image: "registry.example.com/tsuru/app-x:v1", repo: "tsuru/app-x", region: ""},
{image: "app-hello-world", err: true},
}
for _, tt := range tests {
repo, region, err := parseImage(tt.image)
if tt.err {
assert.Error(t, err, tt.image)
continue
}
assert.NoError(t, err, tt.image)
assert.Equal(t, tt.repo, repo, tt.image)
assert.Equal(t, tt.region, region, tt.image)
}
}
3 changes: 3 additions & 0 deletions pkg/repository/repository.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (
"encoding/json"
"fmt"

"github.com/tsuru/deploy-agent/pkg/repository/ecr"
"github.com/tsuru/deploy-agent/pkg/repository/fake"
"github.com/tsuru/deploy-agent/pkg/repository/oci"
)
Expand All @@ -23,6 +24,8 @@ func repositoryProvider(providerType string, data map[string]string) (Repository
switch providerType {
case "oci":
return oci.NewOCI(data), nil
case "ecr":
return ecr.NewECR(data), nil
case "fake":
return &fake.FakeRepository{}, nil
default:
Expand Down
8 changes: 7 additions & 1 deletion pkg/repository/repository_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"testing"

"github.com/stretchr/testify/assert"
"github.com/tsuru/deploy-agent/pkg/repository/ecr"
"github.com/tsuru/deploy-agent/pkg/repository/fake"
"github.com/tsuru/deploy-agent/pkg/repository/oci"
)
Expand All @@ -21,15 +22,20 @@ func TestNewRemoteRepository(t *testing.T) {
},
"faker.com": {
"provider": "fake"
},
"123456789012.dkr.ecr.us-east-1.amazonaws.com": {
"provider": "ecr",
"region": "us-east-1"
}
}`)
repositoryMap, err := NewRemoteRepository(data)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
assert.Len(t, repositoryMap, 2)
assert.Len(t, repositoryMap, 3)
assert.Equal(t, oci.NewOCI(map[string]string{"compartmentID": "123", "profile": "dev"}), repositoryMap["test.com"])
assert.Equal(t, &fake.FakeRepository{}, repositoryMap["faker.com"].(*fake.FakeRepository))
assert.Equal(t, ecr.NewECR(map[string]string{"provider": "ecr", "region": "us-east-1"}), repositoryMap["123456789012.dkr.ecr.us-east-1.amazonaws.com"])
}

func TestNewRepositoryInvalidProvider(t *testing.T) {
Expand Down
Loading