Skip to content
This repository was archived by the owner on Apr 2, 2026. It is now read-only.
This repository was archived by the owner on Apr 2, 2026. It is now read-only.

table_catalog error when deploying on AWS Lambda #97

Description

@dvd3v

I'm trying to run TiMVT in AWS Lambda + Aurora (based on https://github.com/developmentseed/serverless-timvt).
This is how my application looks like:

from timvt.db import close_db_connection, connect_to_db, register_table_catalog
from timvt.factory import VectorTilerFactory
from timvt.layer import FunctionRegistry
from fastapi import FastAPI, Request

# Create Application.
app = FastAPI()

# Add Function registry to the application state
app.state.timvt_function_catalog = FunctionRegistry()


# Register Start/Stop application event handler to setup/stop the database connection
# and populate `app.state.table_catalog`
@app.on_event("startup")
async def startup_event():
    """Application startup: register the database connection and create table list."""
    await connect_to_db(app)
    await register_table_catalog(
        app
    )


@app.on_event("shutdown")
async def shutdown_event():
    """Application shutdown: de-register the database connection."""
    await close_db_connection(app)

# Register endpoints.
mvt_tiler = VectorTilerFactory(
    with_tables_metadata=True,
    with_functions_metadata=True,  # add Functions metadata endpoints (/functions.json, /{function_name}.json)
    with_viewer=True,
)
app.include_router(mvt_tiler.router, tags=["Tiles"])

Note; i've added the register_table_catalog function so it loads the tables from the db.
Everything works fine locally, also using the Aurora database. However when I deploy the application to AWS Lambda I run into a few problems. I'm using the geolambda layers from https://github.com/lambgeo/geo-layer.

When using Python 3.8 with GDAL 3.2, all the dependencies load correctly, and I'm able to run the app. However when I call the /tables.json endpoint I run into the following error:

[ERROR]	2022-10-12T13:15:46.051Z	9e67c286-5679-44cd-b502-b709978cea4b	An error occurred running the application.
Traceback (most recent call last):
  File "/var/task/starlette/datastructures.py", line 699, in __getattr__
    return self._state[key]
KeyError: 'table_catalog'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/task/mangum/protocols/http.py", line 58, in run
    await app(self.scope, self.receive, self.send)
  File "/var/task/fastapi/applications.py", line 270, in __call__
    await super().__call__(scope, receive, send)
  File "/var/task/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/var/task/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/var/task/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/var/task/starlette/middleware/exceptions.py", line 75, in __call__
    raise exc
  File "/var/task/starlette/middleware/exceptions.py", line 64, in __call__
    await self.app(scope, receive, sender)
  File "/var/task/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/var/task/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/var/task/starlette/routing.py", line 680, in __call__
    await route.handle(scope, receive, send)
  File "/var/task/starlette/routing.py", line 275, in handle
    await self.app(scope, receive, send)
  File "/var/task/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/var/task/fastapi/routing.py", line 231, in app
    raw_response = await run_endpoint_function(
  File "/var/task/fastapi/routing.py", line 160, in run_endpoint_function
    return await dependant.call(**values)
  File "/var/task/timvt/factory.py", line 232, in tables_index
    for r in request.app.state.table_catalog
  File "/var/task/starlette/datastructures.py", line 702, in __getattr__
    raise AttributeError(message.format(self.__class__.__name__, key))
AttributeError: 'State' object has no attribute 'table_catalog'

When using a Python 3.7 or 3.9 environment, the application does not load and returns the following error:

[ERROR] Runtime.ImportModuleError: Unable to import module 'handler': No module named 'asyncpg.protocol.protocol'
Traceback (most recent call last):

This could be related to the c-bindings of the asyncpg package.

I cannot reproduce these errors locally, so it has something to do with the AWS Lambda runtime environment.
Any ideas?

Activity

  1. changed the title [-]table_catalog error when deploying TiMVT on AWS Lambda[/-] [+]table_catalog error when deploying on AWS Lambda[/+] on Oct 12, 2022
  2. vincentsarago commented on Oct 12, 2022

    @vincentsarago
    Member

    @dvd3v do you have a link to the code you are using?

    This could be a issue with your handler. If it's not the case try setting lifespan="auto" see: https://github.com/developmentseed/eoAPI/blob/master/stack/handlers/vector_handler.py#L12

    When using a Python 3.7 or 3.9 environment, the application does not load and returns the following error:

    This is a dependency issue, and could be because of a lot of things.

  3. dvd3v commented on Oct 12, 2022

    @dvd3v
    Author

    Here's the code I'm using: https://github.com/dvd3v/serverless_timvt (note that the Aurora DB is not in here as it was setup in a different project.)

    When setting the lifespan="auto", I get timeout errors. Setting it to off works and them I'm able to call the endpoints.

    Regarding the table_catalog error, I've noticed this only happens when installing TiMVT via pip (pip install timvt). When installing from source (pip install git+https://github.com/developmentseed/timvt.git#egg=timvt), the error does not occur. So it's related to timvt 0.7.0.
    However, the /tables.json endpoint then returns emtpy values:

    Expected:

    [
      {
        "id": "public.sentinel_mgrs",
        "bounds": [
          -180,
          -83.83595275878906,
          180,
          83.74835205078125
        ],
        "minzoom": 0,
        "maxzoom": 22,
        "default_tms": "WebMercatorQuad",
        "tileurl": "http://127.0.0.1:8003/tiles/public.sentinel_mgrs/{z}/{x}/{y}",
        "type": "Table",
        "schema": "public",
        "table": "sentinel_mgrs",
        "geometry_type": "POLYGON",
        "geometry_column": "geom",
        "geometry_srid": 4326,
        "properties": {
          "id": "varchar",
          "geom": "geometry",
          "name": "varchar",
          "ogc_fid": "int4"
        }
      }
    ]

    Return from Lambda:

    []

    The Lambda has access to the DB and locally it returns the correct data.

  4. vincentsarago commented on Oct 12, 2022

    @vincentsarago
    Member

    there are a lot of change that haven't been published yet https://github.com/developmentseed/timvt/blob/master/CHANGES.md#next-tbd

  5. locked and limited conversation to collaborators on Oct 12, 2022
  6. converted this issue into a discussion #98 on Oct 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions