[Bug] ssl_verify parameter for IndexAsyncio does not work

[williamhakim10]

Is this a new bug?
In other words: Is this an error, flaw, failure or fault? Please search Github issues and check our Community Forum to see if someone has already reported the bug you encountered.

If this is a request for help or troubleshooting code in your own Pinecone project, please join the Pinecone Community Forum.

• I believe this is a new bug
• I have searched the existing Github issues and Community Forum, and I could not find an existing post for this bug

Describe the bug

When instantiating a new async index with ssl_verify=False, e.g.:

async with pc.IndexAsyncio(
    host="localhost:5081", name="test-index", ssl_verify=False
) as idx:
    yield idx

API calls (specifically update, but probably all the others) do not work because aiohttp TCP Connectors require verify_ssl or ssl_context but not both. See https://github.com/aio-libs/aiohttp/blob/v3.9.0/aiohttp/client_reqrep.py#L169. The solution probably looks something like this, see in https://github.com/pinecone-io/pinecone-python-client/blob/cf20e5d8b12b88fb3805511a5272a2f0ad054894/pinecone/openapi_support/rest_aiohttp.py#L17

if verify_ssl:
        if configuration.ssl_ca_cert is not None:
            ca_certs = configuration.ssl_ca_cert
        else:
            ca_certs = certifi.where()

        ssl_context = ssl.create_default_context(cafile=ca_certs)
        conn = aiohttp.TCPConnector(verify_ssl=True, ssl=ssl_context)
else:
        conn = aiohttp.TCPConnector(verify_ssl=False)

You could also use a more modern version of aiohttp which doesn't have these legacy ssl parameters.

Error information

../embeddings/pinecone_writer.py:170: in update_metadata
    async with self.index_async(model_id=model_id, catalog=catalog_id) as idx:
../../../../.pyenv/versions/3.10.13/lib/python3.10/contextlib.py:199: in __aenter__
    return await anext(self.gen)
../embeddings/pinecone_writer.py:147: in index_async
    async with self.__client.IndexAsyncio(
../../../../Library/Caches/pypoetry/virtualenvs/embeddings-A5ycbq_j-py3.10/lib/python3.10/site-packages/pinecone/control/pinecone.py:332: in IndexAsyncio
    return _IndexAsyncio(
../../../../Library/Caches/pypoetry/virtualenvs/embeddings-A5ycbq_j-py3.10/lib/python3.10/site-packages/pinecone/data/index_asyncio.py:150: in __init__
    self._vector_api = setup_async_openapi_client(
../../../../Library/Caches/pypoetry/virtualenvs/embeddings-A5ycbq_j-py3.10/lib/python3.10/site-packages/pinecone/utils/setup_openapi_client.py:12: in setup_async_openapi_client
    api_client = api_client_klass(configuration=openapi_config)
../../../../Library/Caches/pypoetry/virtualenvs/embeddings-A5ycbq_j-py3.10/lib/python3.10/site-packages/pinecone/openapi_support/asyncio_api_client.py:39: in __init__
    self.rest_client = AiohttpRestClient(configuration)
../../../../Library/Caches/pypoetry/virtualenvs/embeddings-A5ycbq_j-py3.10/lib/python3.10/site-packages/pinecone/openapi_support/rest_aiohttp.py:24: in __init__
    conn = aiohttp.TCPConnector(verify_ssl=configuration.verify_ssl, ssl=ssl_context)
../../../../Library/Caches/pypoetry/virtualenvs/embeddings-A5ycbq_j-py3.10/lib/python3.10/site-packages/aiohttp/connector.py:867: in __init__
    self._ssl = _merge_ssl_params(ssl, verify_ssl, ssl_context, fingerprint)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

ssl = <ssl.SSLContext object at 0x131598ac0>, verify_ssl = False
ssl_context = None, fingerprint = None

    def _merge_ssl_params(
        ssl: Union["SSLContext", bool, Fingerprint],
        verify_ssl: Optional[bool],
        ssl_context: Optional["SSLContext"],
        fingerprint: Optional[bytes],
    ) -> Union["SSLContext", bool, Fingerprint]:
        if ssl is None:
            ssl = True  # Double check for backwards compatibility
        if verify_ssl is not None and not verify_ssl:
            warnings.warn(
                "verify_ssl is deprecated, use ssl=False instead",
                DeprecationWarning,
                stacklevel=3,
            )
            if ssl is not True:
>               raise ValueError(
                    "verify_ssl, ssl_context, fingerprint and ssl "
                    "parameters are mutually exclusive"
                )
E               ValueError: verify_ssl, ssl_context, fingerprint and ssl parameters are mutually exclusive

Environment

• OS Version: MacOS Sonoma 14.3
• Python version: Python 3.10.13
• Python SDK version: 6.0.1

[mcpaddy]

Hello @williamhakim10,

Thank you for raising the bug report. Pinecone Local does not support TLS/SSL. This means that when you connect to it from one of the Pinecone SDKs, the protocol needs to be set in the host parameter, like so:

async with pc.IndexAsyncio(
    host="http://localhost:5081", name="test-index"
) as idx:
    yield idx

That said, I do think this could be a valid bug when using a proxy, where one might need to disable certificate validation.

[williamhakim10]

Hi @mcpaddy - thanks for your response. Your example does indeed work. The confusing part (for us) is that we have code like this:

host = pc.describe_index(index_name).host
async with pc.IndexAsyncio(host=host, ...) as idx:
    yield idx

Normally this works fine, because describe_index returns a host without a scheme, and any host without a scheme is assumed to be https:// by the index. However, in the case of Pinecone Local, SSL isn't supported, so you get an SSL error.

[arthurgousset]

Hey @jhamon, I stumbled over this issue and looked into it out of curiosity. I have a PR (https://github.com/hyperdrive-eng/pinecone-python-client/pull/1) that reproduces the bug and adds a fix (effectively what @williamhakim10 described above). Are you open to a contribution here? Totally fine if not, just thought I'd ask before opening a PR.