Scala Native HTTP

Scala Native implementation for java.net.http.

This project is forked from lolgab/scala-native-http-client-async and restarted due to com-lihaoyi/requests-scala#156 and scala-native/scala-native#4104

Still working in progress, any kind of help is welcome and appreciated.

java.net.http provides Provides high-level client interfaces to HTTP (versions 1.1 and 2) and low-level client interfaces to WebSocket. There are four main types defined:

• HttpClient
• HttpRequest
• HttpResponse
• WebSocket

The development of first three types is the milestone in current stage.

Besides, TLS support is also a important part of HTTP Client, so we also implement javax.net.ssl related types as well.

• javax.net.ssl.SSLContext

There are also other auxiliary types are enforced by the Java Networking API and are missing in Scala Native core, such as:

1. Vendored by dependencies
   • java.time.Duration
     • Provided by cquiroz/scala-java-time
   • java.util.Locale
     • Provided by cquiroz/scala-java-locales
2. Will try to include in this package
   • java.net.Proxy
   • java.net.ProxySelector
   • java.net.Authenticator
   • java.net.CookieHandler

Classes list for Java Http Client shims

Classes don't have checkbox mean they are implemented in Scala Native itself. Classes with checkbox and are checked mean they are implemented in this project.

• java.net
  • Authenticator
  • CacheRequest
  • CacheResponse
  • ContentHandler
  • ContentHandlerFactory
  • CookieHandler
  • CookieManager
  • CookiePolicy
  • CookieStore
  • FileNameMap
  • HttpCookie
  • HttpURLConnection
  • JarURLConnection
  • PasswordAuthentication
  • Proxy
  • ProxySelector
  • ResponseCache
  • SecureCacheResponse
  • URI
    • included in Scala Native already
  • URL
  • URIEncoderDecoder
    • included in Scala Native already
  • URLDecoder
    • included in Scala Native already
  • URLEncoder
    • included in Scala Native already
  • URLConnection
  • URLStreamHandler
  • URLStreamHandlerFactory
  • Exceptions
    • HttpRetryException
    • URISyntaxException (included in Scala Native, others in similar level are the same)
    • MalformedURLException
    • ProtocolException
    • UnknownHostException
    • UnknownServiceException
• java.net.http
  • HttpClient
  • HttpHeaders
  • HttpRequest
  • HttpResponse
  • WebSocket
  • Exceptions
    • HttpConnectTimeoutException
    • HttpTimeoutException
    • WebSocketHandshakeException
• javax.net
  • ServerSocketFactory
  • SocketFactory
  • ssl
    • CertPathTrustManagerParameters
    • KeyManagerFactory
    • SNIHostName
    • SNIMatcher
    • SNIServerName
    • SSLContext
    • SSLEngine
      • partially implemented, only client side is supported
    • SSLEngineResult
    • SSLException
    • SSLParameters
    • SSLSession
    • SSLSessionContext
    • TrustManager
    • X509TrustManager
    • exceptions

Dev guides

This project is developed in Scala 3 and Scala Native 0.5.7 with mill build tool.

Please ensure you have libcurl installed.

• For Linux user with Debian / Ubuntu, the package name would be libcurl4-openssl-dev.
• For macOS user, install curl by Homebrew first.
• I haven't tested on other OS, you may need tune the includeLibCurl config in build.mill file.

In the current codebase, most implementations are layouted under modules/java-http-client, which will be published as scala-native-java-http. And the the binding for curl is included under modules/java-http-client/src/snhttp/experimental/libcurl. I will try to split to different sub modules in the future.

Serveral convenient recipes have been provided by Makefile, for example

• make bsp: Install bsp by mill
• make install-mill-autocomp: Run mill.tabcomplete/install and source the generated completions.
• make fmt: Run scalafmt over codebase
• make compile: Compile major codes
• make test: Run tests

Check Makefile for more.

Known Issues

Fixable (not supported yet)

• Client session resumption
• Custom SSL providers
• HTTP/1.1 connection pooling
• java.net.IDN is not implemented yet, so non-ASCII hostnames (ponycode) may not work as expected
  • influence SNIHostName then SNIMatcher
• Streaming responses

Unfixable

Properties support

Not all HTTP Client properties listed in

• Java Networking Docs
• Module java.net.http - System properties used by the java.net.http API

are well supported, but we may try to implement them in the future. Currently supported properties are:

• java.net.useSystemProxies
• jdk.httpclient.allowRestrictedHeaders
• jdk.httpclient.auth.retrylimit
• jdk.httpclient.bufsize
• jdk.httpclient.connectionPoolSize
• jdk.httpclient.connectionWindowSize
• jdk.httpclient.disableRetryConnect
• jdk.httpclient.enableAllMethodRetry
• jdk.httpclient.enablepush
• jdk.httpclient.hpack.maxheadertablesize
• jdk.httpclient.HttpClient.log
• jdk.httpclient.keepalive.timeout
• jdk.httpclient.keepalive.timeout.h2
• jdk.httpclient.maxframesize
• jdk.httpclient.maxLiteralWithIndexing
• jdk.httpclient.maxNonFinalResponses
• jdk.httpclient.maxstreams
• jdk.httpclient.receiveBufferSize
• jdk.httpclient.redirects.retrylimit
• jdk.httpclient.sendBufferSize
• jdk.httpclient.websocket.writeBufferSize
• jdk.httpclient.windowsize
• jdk.internal.httpclient.disableHostnameVerification
• jdk.tls.client.SignatureSchemes
• jdk.tls.server.SignatureSchemes
• jdk.tls.namedGroups

Future

I'm thinking curl is probably not the minimal component for HTTP Client, so I may play to create serveral other bindings for Scala Native HTTP libraries, such as:

• scala-native-brotli package for google/brotli
• scala-native-ada package for ada-url/ada
• nghttp2/nghttp2
• ngtcp2/nghttp3

That's should provide a more flexible , extensible HTTP Client for Scala Native with composability.