Please, help me.

When I use the ktor-build-plugin to execute the shadowJar task, the compiler gives me the following error.

> Task :service:service-api:shadowJar FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':service:service-api:shadowJar'.
> shadow.org.apache.tools.zip.Zip64RequiredException: archive contains more than 65535 entries.

  To build this archive, please enable the zip64 extension.
  See: https://docs.gradle.org/8.5/dsl/org.gradle.api.tasks.bundling.Zip.html#org.gradle.api.tasks.bundling.Zip:zip64

Perhaps it's just an issue of documentation, but when I enable logging with HttpClient(Android) no logs is present unless I provide custom logger implementation.

In more details:

When I create HttpClient with Android Engine using either Logger.DEFAULT or Logger.ANDROID no logs is present in the Logcat .

    HttpClient(Android) {
        if (SdkConfig.isLoggingEnabled) {
            install(Logging) {
                logger = Logger.DEFAULT
                level = LogLevel.ALL
            }
        }
    }

I am adding logging dependency as io.ktor:ktor-client-logging-jvm:1.4.1. As far as I see there is no specific "android" artefact available.

As far as I see both Logger.DEFAULT or Logger.ANDROID uses slf4j for logging, which does nothing on Android by default.

Ideally when Logger.DEFAULT logger is used in Android it should log to Android system log.

And obviously Logger.ANDROID is expected to do this without any further configuration.

P.S. I have not tried yet logging on JVM, but as for me, it make sense to add the mentioning about required slf4j implementation (e.g. logback) in the dependencies there as well.

When the server immediately sends a frame back to the client upon handshake, the client sometimes misses it.

Possible root cause

I believe this is due to the fact that the onMessage web socket listener is not registered immediately after the creation of the web socket instance (which performs the handshake). In JS, since things are single-threaded, it's OK to do the following:

val ws = WebSocket(url)
ws.onmessage = { ... }

But in the case of Ktor implementation, there is an awaitConnection() between the handshake and the message listener registration:
https://github.com/ktorio/ktor/blob/2.3.9/ktor-client/ktor-client-core/js/src/io/ktor/client/engine/js/JsClientEngine.kt#L91-L100

val socket: WebSocket = createWebSocket(urlString, request.headers) // performs the handshake

try {
    socket.awaitConnection()
} catch (cause: Throwable) {
    callContext.cancel(CancellationException("Failed to connect to $urlString", cause))
    throw cause
}

val session = JsWebSocketSession(callContext, socket) // registers socket.onmessage in init{} block

I think the presence of this await here allows a frame to arrive before the message event listener is registered, thus dropping the frame.

Reproducer

First, we need to setup a test server that sends a frame upon handshake. For instance, this simple test server uses org.java-websocket:Java-WebSocket:1.5.6:

import org.java_websocket.*
import org.java_websocket.handshake.*
import org.java_websocket.server.*
import java.net.*
import java.nio.*

fun main() {
    val port = 12345
    val server = EchoWebSocketServer(port)
    println("Starting test WS server...")
    server.start()
    println("Server listening at ws://localhost:$port")
}

internal class EchoWebSocketServer(port: Int = 0) : WebSocketServer(InetSocketAddress(port)) {

    override fun onOpen(conn: WebSocket, handshake: ClientHandshake) {
        println("Received connection from ${conn.remoteSocketAddress}${conn.resourceDescriptor}")
        conn.send("Hello client!")
        println("Said hi to the client")
    }

    override fun onStart() {}
    override fun onMessage(conn: WebSocket, message: String?) {}
    override fun onMessage(conn: WebSocket, message: ByteBuffer) {}
    override fun onError(conn: WebSocket?, ex: Exception?) {}
    override fun onClose(conn: WebSocket, code: Int, reason: String?, remote: Boolean) {}
}

Then we can run this test that demonstrates the problem:

import io.ktor.client.*
import io.ktor.client.plugins.websocket.*
import io.ktor.websocket.*
import kotlinx.coroutines.*
import kotlinx.coroutines.test.*
import kotlin.test.*
import kotlin.time.Duration.Companion.minutes
import kotlin.time.Duration.Companion.seconds

private const val port = 12345 // TODO replace with the acutal port used by the test server
private const val nAttempts = 10 // usually the problem occurs after 2-3 iterations
private val frameTimeout = 2.seconds // we can increase this to confirm it's not a matter of time

class SomeJsTest {
    @Test
    fun testHandshakeResponse() = runTest(timeout = 1.minutes) {
        withContext(Dispatchers.Default) { // to avoid delay skipping from runTest (we're testing a real system)
            var nMissed = 0
            repeat(nAttempts) { attempt ->
                val httpClient = HttpClient { install(WebSockets) }
                val session = httpClient.webSocketSession("ws://localhost:$port")

                try {
                    println("Expecting immediate hello from the server...")
                    val helloFrame = withTimeoutOrNull(frameTimeout) {
                        session.incoming.receive()
                    }
                    if (helloFrame == null) {
                        println("$attempt: MISSED! (Hello frame not received $frameTimeout after handshake)")
                        nMissed++
                    } else {
                        assertIs<Frame.Text>(helloFrame, "should receive a Text frame as echo")
                        assertEquals("Hello client!", helloFrame.readText(), "should receive echo")
                        println("$attempt: Got our hello frame this time!")
                    }
                } finally {
                    session.close()
                }
            }
            if (nMissed > 0) fail("$nMissed/$nAttempts frames were missed")
        }
    }
}

The output should look like this:

Expecting immediate hello from the server...
0: Got our hello frame this time!
Expecting immediate hello from the server...
1: MISSED! (Hello frame not received 2s after handshake)
Expecting immediate hello from the server...
2: MISSED! (Hello frame not received 2s after handshake)
Expecting immediate hello from the server...
3: Got our hello frame this time!
Expecting immediate hello from the server...
4: Got our hello frame this time!
Expecting immediate hello from the server...
5: Got our hello frame this time!
Expecting immediate hello from the server...
6: Got our hello frame this time!
Expecting immediate hello from the server...
7: MISSED! (Hello frame not received 2s after handshake)
Expecting immediate hello from the server...
8: MISSED! (Hello frame not received 2s after handshake)
Expecting immediate hello from the server...
9: MISSED! (Hello frame not received 2s after handshake)

AssertionError: 5/10 frames were missed

Setting target to es2015 causes JS client to fail in Chrome

Reproducer:

1. Create a minimal project with the code below
2. Run tests, observe that the tests succeed
3. Run jsBrowserDevelopmentWebpack
4. Create this html file

<body style="background: black; color: white">
<p>hello world</p>
<script src="/path/to/ktor-bug-project/build/kotlin-webpack/js/developmentExecutable/ktor-bug-project.js"></script>
</body>

5. Open this HTML file in Google Chrome and observe the error

build.gradle.kts

plugins {
    kotlin("multiplatform") version "2.0.0-Beta5"
}
repositories.mavenCentral()
kotlin {
    js(IR) {
        @Suppress("OPT_IN_USAGE")
        compilerOptions.target.set("es2015")
        browser{
            binaries.executable()
        }
    }
    sourceSets.jsMain.dependencies {
        implementation("io.ktor:ktor-client-js:2.3.9")
    }
}
dependencies {
    commonTestImplementation(kotlin("test"))
}

main.kt

import io.ktor.client.HttpClient
import io.ktor.client.request.get
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Job
import kotlinx.coroutines.launch

fun main() {
    CoroutineScope(Job()).launch {
        HttpClient().get("https://www.example.com")
    }
}

test.kt

import kotlin.test.Test

class Tests {
    @Test
    fun runMain() {
        main()
    }
}

My host have only ip and self-signed certificate. My version ktor 2.3.8
My client setup:

client = HttpClient(CIO) {
            install(WebSockets){
                pingInterval = 5_000
            }
            engine {
                https {
                    trustManager = object : X509TrustManager {
                        override fun checkClientTrusted(chain: Array<out X509Certificate>?, authType: String?) { }
                        override fun checkServerTrusted(chain: Array<out X509Certificate>?, authType: String?) { }
                        override fun getAcceptedIssuers(): Array<X509Certificate> {
                            return emptyArray()
                        }
                    }
                }
            }

            install(Logging) {
                logger = object : Logger {
                    override fun log(message: String) {
                        Timber.tag("Socket Log").e(message)
                    }
                }
                level = LogLevel.ALL
            }

        }

But after try connect i have error: certificate.subjectAlternativeNames must not be null

java.lang.NullPointerException: certificate.subjectAlternativeNames must not be null
  at io.ktor.network.tls.HostnameUtilsKt.verifyIpInCertificate(HostnameUtils.kt:30)
  at io.ktor.network.tls.HostnameUtilsKt.verifyHostnameInCertificate(HostnameUtils.kt:15)
  at io.ktor.network.tls.TLSClientHandshake.handleCertificatesAndKeys(TLSClientHandshake.kt:245)
  at io.ktor.network.tls.TLSClientHandshake.negotiate(TLSClientHandshake.kt:166)
  at io.ktor.network.tls.TLSClientHandshake$negotiate$1.invokeSuspend(Unknown Source:18)
  at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
  at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
  at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
  at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
  at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
  Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@abda18f, Dispatchers.IO]

Use this code to compare between Netty and CIO and visit http://localhost:8080/?auto, http://localhost:8081/?auto`:

fun main() {
    embeddedServer(Netty, port = 8080) {
        routing {
            get("/") {
                call.respondText(this.context.request.queryParameters["auto"].toString())
            }
        }
    }.start()

    embeddedServer(CIO, port = 8081) {
        routing {
            get("/") {
                call.respondText(this.context.request.queryParameters["auto"].toString())
            }
        }
    }.start(true)
}

CIO sets the query parameter to null (so it's equivalent to auto not being present at all). Netty sets it to a blank string (so you can detect its presence):

cio-qpm.zip

To reproduce, run the following test:

@Test
fun test() = testApplication {
    application {
        install(CallLogging)
        install(StatusPages) {
            status(HttpStatusCode.BadRequest) { call, _ ->
                call.respondText { "Bad request" }
            }
        }

        routing {
            get {
                call.respond(HttpStatusCode.BadRequest)
            }
        }
    }

    client.get("/")
}

As a result, the response for the / request is unexpectedly logged twice:

10:01:25.843 [DefaultDispatcher-worker-1 @request#2] INFO  io.ktor.test -- 200 OK: GET - / in 137ms
10:01:25.843 [DefaultDispatcher-worker-1 @request#2] INFO  io.ktor.test -- 200 OK: GET - / in 137ms

If you access the server through the IPv6 address, it gives an error

exception: java.lang.NumberFormatException: For input string: "ff:ff:ff:ff:ff:ff:ff]:8080"
	at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
	at java.base/java.lang.Integer.parseInt(Integer.java:668)
	at java.base/java.lang.Integer.parseInt(Integer.java:786)
	at io.ktor.server.netty.http1.NettyConnectionPoint.getServerPort(NettyConnectionPoint.kt:55)
	at io.ktor.server.routing.HostRouteSelector.evaluate(HostsRoutingBuilder.kt:111)
	at io.ktor.server.routing.RoutingResolveContext.handleRoute(RoutingResolveContext.kt:97)
	at io.ktor.server.routing.RoutingResolveContext.handleRoute(RoutingResolveContext.kt:157)
	at io.ktor.server.routing.RoutingResolveContext.resolve(RoutingResolveContext.kt:82)
	at io.ktor.server.routing.Routing.interceptor(Routing.kt:62)
	at io.ktor.server.routing.Routing$Plugin$install$1.invokeSuspend(Routing.kt:140)

NettyConnectionPoint contains an error. When trying to get the server port or server host, it splits the string at the first occurrence of the ":" character. But you can access the server using the IPv6 address, so it's better to cut off by the last occurrence of the ":" character (substringAfterLast)

NettyConnectionPoint.kt error place:

override val serverPort: Int
        get() = request.headers().get(HttpHeaders.Host)
            ?.substringAfter(":", defaultPort.toString())
            ?.toInt()
            ?: localPort

Tried to upload a large-ish file after recently trying out CIO, now faced with this problem on a 12 GiB file:

an error has happened: For input string: "12255718583"
java.lang.NumberFormatException: For input string: "12255718583"
	at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.base/java.lang.Integer.parseInt(Integer.java:652)
	at java.base/java.lang.Integer.parseInt(Integer.java:770)
	at io.ktor.server.cio.CIOApplicationEngine.hasBody(CIOApplicationEngine.kt:153)
	at io.ktor.server.cio.CIOApplicationEngine.access$hasBody(CIOApplicationEngine.kt:24)
	at io.ktor.server.cio.CIOApplicationEngine$addHandlerForExpectedHeader$2.invokeSuspend(CIOApplicationEngine.kt:135)
	at io.ktor.server.cio.CIOApplicationEngine$addHandlerForExpectedHeader$2.invoke(CIOApplicationEngine.kt)
	at io.ktor.server.cio.CIOApplicationEngine$addHandlerForExpectedHeader$2.invoke(CIOApplicationEngine.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.request.ApplicationReceiveFunctionsKt.receiveNullable(ApplicationReceiveFunctions.kt:103)
	at io.sebi.ui.AddUploadKt$addUpload$2.invokeSuspend(AddUpload.kt:113)
	at io.sebi.ui.AddUploadKt$addUpload$2.invoke(AddUpload.kt)
	at io.sebi.ui.AddUploadKt$addUpload$2.invoke(AddUpload.kt)
	at io.ktor.server.routing.Route$buildPipeline$1$1.invokeSuspend(Route.kt:116)
	at io.ktor.server.routing.Route$buildPipeline$1$1.invoke(Route.kt)
	at io.ktor.server.routing.Route$buildPipeline$1$1.invoke(Route.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
	at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invoke(Pipeline.kt)
	at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invoke(Pipeline.kt)
	at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
	at io.ktor.server.routing.Routing.executeResult(Routing.kt:190)
	at io.ktor.server.routing.Routing.interceptor(Routing.kt:64)
	at io.ktor.server.routing.Routing$Plugin$install$1.invokeSuspend(Routing.kt:140)
	at io.ktor.server.routing.Routing$Plugin$install$1.invoke(Routing.kt)
	at io.ktor.server.routing.Routing$Plugin$install$1.invoke(Routing.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invokeSuspend(BaseApplicationEngine.kt:124)
	at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invoke(BaseApplicationEngine.kt)
	at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invoke(BaseApplicationEngine.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.server.application.hooks.CallFailed$install$1$1.invokeSuspend(CommonHooks.kt:45)
	at io.ktor.server.application.hooks.CallFailed$install$1$1.invoke(CommonHooks.kt)
	at io.ktor.server.application.hooks.CallFailed$install$1$1.invoke(CommonHooks.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:61)
	at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:261)
	at io.ktor.server.application.hooks.CallFailed$install$1.invokeSuspend(CommonHooks.kt:44)
	at io.ktor.server.application.hooks.CallFailed$install$1.invoke(CommonHooks.kt)
	at io.ktor.server.application.hooks.CallFailed$install$1.invoke(CommonHooks.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
	at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invokeSuspend(DefaultEnginePipeline.kt:118)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invoke(DefaultEnginePipeline.kt)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invoke(DefaultEnginePipeline.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
	at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
	at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
	at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
	at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2.invokeSuspend(CIOApplicationEngine.kt:238)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
	at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:111)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:99)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:585)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:802)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:706)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:693)

To reproduce make a / request to the following server:

embeddedServer(Netty, port = 3000) {
    install(CallLogging) {
        mdc("hardcoded") {
            "1"
        }
    }
    routing {
        val r = route("/") {
            handle {
                call.respondText { "OK" }
            }
        }

        r.intercept(ApplicationCallPipeline.Setup) {
            MDC.put("name", "<name>")

            withContext(MDCContext()) {
                proceed()
            }
        }
    }
}.start(wait = true)

logback.xml:

<configuration>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>[%X{name}] %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%kvp- %msg%n</pattern>
        </encoder>
    </appender>

    <root level="debug">
        <appender-ref ref="STDOUT" />
    </root>
</configuration>

Expected: the MDC key name has value <name> in the logs.
Actual: the MDC key name has an empty value in the logs.

Responses from SSE server contains duplicate headers:

HTTP/1.1 200 OK
Vary: Origin
Content-Type: text/event-stream <-------
Cache-Control: no-store
Connection: keep-alive
Content-Type: text/event-stream <-------
transfer-encoding: chunked

This causes the following exception in the JS client:

SSEException: Expected Content-Type text/event-stream but was: text/event-stream, text/event-stream

This comes from installing the SSE plugin on the client/server and sending an sse response as follows:

sse {
    send("Hello")
}

If I add a header("Accept", "application/json") at request builder block, the content negotiation plugin will add another one，then the header became "Accept": "application/json,application/json"

Since March, we haven't been receiving project created metrics from start.ktor.io. This is likely caused from the front-end library updates.

Please consider removing the following exception in ResponseCookies :

if (item.secure && !secureTransport) {
  throw IllegalArgumentException("You should set secure cookie only via secure transport (HTTPS)")
}

When using a reverse proxy we can't set the secure cookie. I know there is the XForwardedHeaderSupport plugin but it is not customizable (https://youtrack.jetbrains.com/issue/KTOR-2657) and not all configurations will match the specification described in XForwardedHeaderSupport.

The web framework should let the developper decide if they want to send a secure cookie or not.

In the meantime, here is a hotfix to disable this exception without using XForwardedHeaderSupport:

intercept(ApplicationCallPipeline.Features) {
    val endpoint = call.attributes.computeIfAbsent(MutableOriginConnectionPointKey) {
        MutableOriginConnectionPoint(call.request.origin)
    }
    endpoint.scheme = "https"
}

Current error description:

io.ktor.client.call.NoTransformationFoundException: Expected response body of the type 'class io.ktor.client.plugins.websocket.DefaultClientWebSocketSession' but was 'class io.ktor.utils.io.ByteBufferChannel'
In response from `https://example.com`
Response status `500 Server Error`
Response header `ContentType: text/xml; charset=UTF-8` 
Request header `Accept: application/xml`

Expected error description:
ServerResponseException with the 500 error and ideally the textual response

When using ktor 2.3.5 client with CIO engine on an Android with minSdk of 25, ktor fails in verifyHostnameInCertificate due to: java.lang.NullPointerException: getSubjectAlternativeNames(...) must not be null. This is for a domain which has a custom certificate authority. The custom ca is added to the Android project as a raw resource according to the Android network security configuration, so we have:

• Certificate in ./app/src/main/res/raw/cert.pem
• AndroidManifest.xml specifying android:networkSecurityConfig="@xml/network_security_config"
• network_security_config.xml with following contents:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
    <base-config cleartextTrafficPermitted="false">
        <trust-anchors>
            <certificates src="@raw/cert" />
            <certificates src="system" />
        </trust-anchors>
    </base-config>
</network-security-config>

The CIO engine X509TrustManager documentation says it should default to system, so I assume the above is all that is needed to support a self-signed certificate authority.

When using ktor 2.3.4, the TLS certificate is loaded correctly and the network connection to the domain is opened, so this would appear to be an issue with 2.3.5.

Here's the full stack trace:

10-19 10:14:22.651 11972 11972 E AndroidRuntime: FATAL EXCEPTION: main
19 10:14:22.651 11972 11972 E AndroidRuntime: Process: com.foo.bar, PID: 11972
-19 10:14:22.651 11972 11972 E AndroidRuntime: java.lang.NullPointerException: getSubjectAlternativeNames(...) must not be null      
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at io.ktor.network.tls.HostnameUtilsKt.hosts(HostnameUtils.kt:92)
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at io.ktor.network.tls.HostnameUtilsKt.verifyHostnameInCertificate(HostnameUtils.kt:19)
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at
io.ktor.network.tls.TLSClientHandshake.handleCertificatesAndKeys(TLSClientHandshake.kt:245)
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at
io.ktor.network.tls.TLSClientHandshake.negotiate(TLSClientHandshake.kt:166)
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at io.ktor.network.tls.TLSClientHandshake$negotiate$1.invokeSuspend(TLSClientHandshake.kt)
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at
kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at
kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)                           
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at
kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)                        
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)               
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)                 
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)                  
10-19 10:14:22.651 11972 11972 E AndroidRuntime:        Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@3b0a279, Dispatchers.Main.immediate]

EDIT: I've attached the public CA file in question.

On React Native based platforms hasNodeApi will return false and so the platform will end up as BROWSER, but window.location will be undefined, so this code crashes when it tries to retrieve the origin: https://github.com/ktorio/ktor/blob/3.0.0-beta-1/ktor-http/js/src/io/ktor/http/URLBuilderJs.kt#L21

When using the logging plugin (io.ktor.client.plugins.logging.Logging) it causes a block on request streaming, but only when the level is set to LogLevel.BODY. It seems it wants to log it all at the end- instead of as it comes in.

val client = HttpClient {
  install(Logging) {
    level = LogLevel.BODY
    logger = Logger.DEFAULT
  }
}

launch {
 client.preparePost("...") {
  val channel = it.bodyAsChannel()
  while (!isClosedForRead) {
    awaitContent() // will not return until the request is complete
  }
 }
}

This is a big issue for any long running streaming endpoints- as you aren't able to disperse the data in real time.

This issue also persists when using MockEngine- which should allow for easier debugging:

val channel = ByteChannel(autoFlush = true)
val mockEngine = MockEngine {
  respond(
    content = channel,
    status = HttpStatusCode.OK
  )
}
val client = HttpClient(mockEngine) {
  install(Logging) {
    level = LogLevel.BODY
    logger = Logger.DEFAULT
  }
}
val content = channelFlow {
  launch {
    client.preparePost("...") {
      val channel = it.bodyAsChannel()
      while (!isClosedForRead) {
        awaitContent()
        send(readUTF8Line())
      }
    }
  }
}

channel.writeFully("Hello world!\n")

withTimeout(5_000) { // the bug will cause this to timeout
  content.collect {
    channel.close()
  }
}

When a valid cache control header like this is passed from a server to the ktor client, an unhandled NumberFormatException occurs

cache-control: max-age=3155695200, public

I want to start multiple web socket clients that send some requests and then close.
For some reason it gets stuck while closing the socket and doesn't terminate.
If webSocketRaw used instead, it terminates as expected
Example to reproduce:

package com.example

import io.ktor.client.*
import io.ktor.client.engine.cio.*
import io.ktor.client.plugins.websocket.*
import io.ktor.client.plugins.websocket.WebSockets
import io.ktor.client.plugins.websocket.cio.*
import io.ktor.http.*
import io.ktor.server.application.*
import io.ktor.server.engine.*
import io.ktor.server.routing.*
import io.ktor.server.websocket.*
import io.ktor.websocket.*
import kotlinx.coroutines.CoroutineStart
import kotlinx.coroutines.channels.consumeEach
import kotlinx.coroutines.joinAll
import kotlinx.coroutines.launch
import kotlinx.coroutines.runBlocking
import java.time.Duration


fun main() {
  val server = embeddedServer(io.ktor.server.cio.CIO, port = 8080) {
    install(io.ktor.server.websocket.WebSockets) {
      pingPeriod = Duration.ofSeconds(15)
      timeout = Duration.ofSeconds(15)
      maxFrameSize = Long.MAX_VALUE
      masking = false
    }
    routing {
      webSocket("/echo") {
        incoming.consumeEach { frame ->
          if (frame is Frame.Text) {
            val receivedText = frame.readText()
            send(Frame.Text("You sent: $receivedText"))
          }
        }
      }
    }
  }.start()
  val CLIENTS_COUNT = 10
  val REQUEST_COUNT = 2000

  runBlocking {
    val jobs = (0 until CLIENTS_COUNT).map { clientId ->
      launch(start = CoroutineStart.LAZY) {
        HttpClient(CIO) {
          install(WebSockets) {
            pingInterval = 20_000
          }
        }.use { client ->
          client.webSocket(method = HttpMethod.Get, host = "127.0.0.1", port = 8080, path = "/echo") {
            for (requestId in 0..REQUEST_COUNT) {
              send(Frame.Text("$clientId-$requestId"))
            }
          }
        }
      }
    }

    jobs.forEach { it.start() }
    jobs.joinAll()
  }
  println("Stopping")
  server.stop(0)
}

So I'm playing around with https://github.com/cashapp/zipline

I'm running Ktor in JS.

Zipline runs the compiled JS in a custom QuickJS engine.

In this case, the PlatformUtilsJs.kt code is picking up that we're a Platform.Browser because process.versions.node does not exist.

Then URLBuilderJs.kt code is trying to fetch the origin from self.location.origin which doesn't exist and errors out.

Here's my stacktrace

E/Zipline: ReferenceError: 'self' is not defined
        at <anonymous> (mnt/agent/work/8d547b974a7be21f/ktor-http/js/src/io/ktor/http/URLBuilderJs.kt:22)
        at get_origin (mnt/agent/work/8d547b974a7be21f/ktor-http/js/src/io/ktor/http/URLBuilderJs.kt:17)
        at Companion_6 (mnt/agent/work/8d547b974a7be21f/ktor-http/common/src/io/ktor/http/URLBuilder.kt:116)
        at Companion_getInstance_9 (ktor-ktor-http.js)
        at URLBuilder (mnt/agent/work/8d547b974a7be21f/ktor-http/common/src/io/ktor/http/URLBuilder.kt:25)
        at HttpRequestBuilder (mnt/agent/work/8d547b974a7be21f/ktor-client/ktor-client-core/common/src/io/ktor/client/request/HttpRequest.kt:65)

I'm invoking this snippet before the URL code as a workaround instead for now

js(
    """
    globalThis.self = {
        location: {},
    };
    """
)

Thanks.

Currently, we have a 'Creating a new Ktor project' tutorial which shows how to create a new project with the IDE plugin. The project structure is examined in each of the following tutorials. In the new tutorial structure, we need to:

• show how to create a project using the generator
• examine the project structure
• show additional tasks to attempt

New content is available here

To reproduce: Generate project using https://start.ktor.io/.

Try to run via ./gradlew run. See zsh: permission denied: ./gradlew.

Workaround: chmod +x gradlew.

Wizard should set correct permissions for the file when generating and zipping the project.

I have a simple call logging use case. When request comes in, we generate a requestId (uuid), then log this requestId for all logging messages that occurred during the processing of this request. This is used for easier debugging of issues, as we return this requestId in the response. Now, the problem is that I seem to get multiple different requestIds for a single call. Below in the sample log you can see there are three different requestId (contained in <>). I would expect all the logs have the same requestId. Am I misunderstanding how this should work, or is this a genuine bug?

I'm running this on Ktor 2.3.5, using Koin 3.5.1 and log4j2 2.21.1 in a kotlin multiplatform project.

Sample log:

2023-11-30 10:03:33,574 INFO  [Application] (eventLoopGroupProxy-4-1 @call-handler#6) <21e7bfa3-06de-4fc2-a1cd-85bda022dae4> GET USER
2023-11-30 10:03:33,775 INFO  [org.kto.database] (eventLoopGroupProxy-4-1 @call-handler#6) <21e7bfa3-06de-4fc2-a1cd-85bda022dae4> Connected to jdbc:postgresql://localhost:5432/test, productName: PostgreSQL, productVersion: 16.0 (Debian 16.0-1.pgdg120+1), logger: org.ktorm.logging.Slf4jLoggerAdapter@33988138, dialect: org.ktorm.support.postgresql.PostgreSqlDialect@2df38877
2023-11-30 10:37:44,912 INFO  [UserService] (eventLoopGroupProxy-4-1 @call-handler#6) <21e7bfa3-06de-4fc2-a1cd-85bda022dae4> IN SERVICE
2023-11-30 10:03:33,999 DEBUG [org.kto.database] (eventLoopGroupProxy-4-1 @call-handler#6) <21e7bfa3-06de-4fc2-a1cd-85bda022dae4> SQL: select <actual_select> 
2023-11-30 10:03:33,999 DEBUG [org.kto.database] (eventLoopGroupProxy-4-1 @call-handler#6) <21e7bfa3-06de-4fc2-a1cd-85bda022dae4> Parameters: <select_params>
2023-11-30 10:03:34,032 DEBUG [org.kto.database] (eventLoopGroupProxy-4-1 @call-handler#6) <21e7bfa3-06de-4fc2-a1cd-85bda022dae4> Results: 0
2023-11-30 10:03:34,048 ERROR [Application] (eventLoopGroupProxy-4-1 @call-handler#6) <526e4baa-694c-44a7-8fd0-fce8a9de3cd2> [/users/10a9c534-30ea-4e2f-bd48-b151a7597991] Unknown exception occurred.Reference requestId: '526e4baa-694c-44a7-8fd0-fce8a9de3cd2'. Message: 'null'
<packages...>.server.exception.UserNotFoundException: null
	at <packages...>.server.service.UserService.getUser(UserService.kt:19) ~[main/:?]
	at <packages...>.server.routing.UserRouteKt$userRoutes$1$2.invokeSuspend(UserRoute.kt:30) ~[main/:?]
2023-11-30 10:03:34,095 INFO  [Application] (eventLoopGroupProxy-4-1 @call-handler#6) <fb918a74-af4b-42f9-b747-caafd368577a> Status: 500 Internal Server Error, HTTP method: GET, User agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36

In the logging example above, it can be seen that the same request had three different requestIds. The correct behaviour should be, that all logs should have the same requestId logged. The CallLogging logs have a different requestId from StatusPages exception handling logs and from the actual route logs...

The code:

// Call logging configuration
install(CallLogging) {
	level = Level.INFO
	mdc("requestId") {
		UUID.randomUUID().toString()
	}
	format { call ->
		val status = call.response.status()
		val httpMethod = call.request.httpMethod.value
		val userAgent = call.request.headers["User-Agent"]
		"Status: $status, HTTP method: $httpMethod, User agent: $userAgent"
	}
}
...
// Exception handling
install(StatusPages) {
	exception<Throwable> { call, cause ->
		val statusCode = HttpStatusCode.InternalServerError
		val requestId = MDC.get("requestId")
		val additionalMessage = "Reference requestId: '${requestId}'. Message: '${cause.message}'"
		val err = Error.UNKNOWN_EXCEPTION.withAdditionalMessage(additionalMessage)

		call.application.log.error("[${call.request.path()}] Unknown exception occurred. $additionalMessage", cause)

		call.respond(statusCode, ErrorResponse(err.errorCode, err.errorMessage, err.additionalMessage))
	}
}
...
// Route config
get("/users/{userId}") {
	call.application.log.info("GET USER")
	val user = userService.getUser(UUID.fromString(call.parameters["userId"]))

	call.respond(HttpStatusCode.OK, user)
}
...
// UserService
private val logger: Logger = LoggerFactory.getLogger(this.javaClass.simpleName)
fun getUser(id: UUID): UserResponse {
	logger.info("IN SERVICE")
	val user = userRepository.findById(id) ?: throw UserNotFoundException(id)
	return mapUserToUserResponse(user)
}
// Log4j2 PatternLayout
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c{3.}] (%t) &lt;%X{requestId}&gt; %msg%n" />

Thank you, Jan

To reproduce, run the following code on an Android emulator:

val keyStoreFile = File("build/keystore.jks")
val keyStore = buildKeyStore {
    certificate("sampleAlias") {
        password = "foobar"
        domains = listOf("127.0.0.1", "0.0.0.0", "localhost")
    }
}
keyStore.saveToFile(keyStoreFile, "123456")

val environment = applicationEngineEnvironment {
    log = LoggerFactory.getLogger("ktor.application")
    connector {
        port = 8080
    }
    sslConnector(
        keyStore = keyStore,
        keyAlias = "sampleAlias",
        keyStorePassword = { "123456".toCharArray() },
        privateKeyPassword = { "foobar".toCharArray() }) {
        port = 8443
        keyStorePath = keyStoreFile
    }
    module {}
}

embeddedServer(Netty, environment).start(wait = true)

As a result the KeyStoreException is thrown:

FATAL EXCEPTION: DefaultDispatcher-worker-1
Process: com.example.ktor_android, PID: 8259
java.security.KeyStoreException: JKS not found
    at java.security.KeyStore.getInstance(KeyStore.java:904)
    at io.ktor.network.tls.certificates.KeyStoreBuilder.build$ktor_network_tls_certificates(builders.kt:160)
    at io.ktor.network.tls.certificates.BuildersKt.buildKeyStore(builders.kt:176)
    at com.example.ktor_android.MainActivity$onCreate$1.invokeSuspend(MainActivity.kt:31)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
    at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
    at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:100)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
    Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@5e4a1a6, Dispatchers.IO]
    Caused by: java.security.NoSuchAlgorithmException: JKS KeyStore not available
        at sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
        at java.security.Security.getImpl(Security.java:628)
        at java.security.KeyStore.getInstance(KeyStore.java:901)
        ... 11 more

When trying to get the ContentNegotiation to specify the charset on JSON responses like so:

install(ContentNegotiation) {
    jackson(contentType = ContentType.Application.Json.withCharset(Charsets.UTF_8))
}

the matching on ingoing requests breaks.

When looking at RequestConverter this is no surprise, since it strips the charset from the requestContentType and then tries to match the full content type provided at registration.

val requestContentType = try {
    call.request.contentType().withoutParameters()
}

...

if (!requestContentType.match(registration.contentType)) {
    LOGGER.trace(
        "Skipping content converter for request type ${receiveType.type} because " +
            "content type $requestContentType does not match ${registration.contentType}"
    )
    return null
}

I guess the charset should also be stripped from the registration content type or there should be another way to specify the charset parameter for responses which does not affect the matching for requests.

I have a "hello, world" Ktor native server (Kotlin/Native on Linux) and have noticed very high memory usage under heavy load. While memory usage starts around 30MB, it climbs to around 9GB when the server is saturated with requests. Memory does fall back to around 125MB after requests have ceased. I've tried different allocators and options but don't see much difference.
Repo for reproducing:
https://github.com/jamesward/ktor-native-server/tree/mem_repro

ApacheBench for request saturation:

ab -c 8 -n 1000000 http://localhost:8080/

I know it's an apples-to-oranges comparison, but running the same thing on the JVM doesn't seem to cause any memory spiking.

I'm not sure if there are optimizations in Ktor for this use case or if it is all on Kotlin/Native.

In Ktor Server 2.1.3, it was possible to have multiple ContentNegotiation "codecs" installed for one Content-Type. During request decoding, the codecs were tried sequentially and the first non-null result was used. In Ktor 2.2.1, this no longer works - only the first codec is tried. This causes issues for example when using org.json and Kotlinx.serialization in parallel (I'm using a custom shim converter that allows me to directly receive JSONObjects).

I suppose the problem appeared in commit https://github.com/ktorio/ktor/commit/be329698362b9c12ca43b5969bdbbcbdb3496081. I don't know the exact mechanism at play here, but I think there is a subtle logic change in the reimplemented convertBody function or its caller.

This demo reproduces the problem:

• Application.kt:

package com.example

import io.ktor.http.ContentType
import io.ktor.server.application.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import io.ktor.server.plugins.contentnegotiation.ContentNegotiation
import io.ktor.server.request.receive
import io.ktor.server.response.respondText
import io.ktor.server.routing.post
import io.ktor.server.routing.routing

fun main() {
    embeddedServer(Netty, port = 8080, host = "0.0.0.0", module = Application::module)
        .start(wait = true)
}

fun Application.module() {
    install(ContentNegotiation) {
        cvtA(ContentType.Application.Json)
        cvtB(ContentType.Application.Json)
    }
    routing {
        post("/") {
            val request = call.receive<CvtBResult>()

            call.respondText("ECHO: " + request.data)
        }
    }
}

• CvtA.kt:

package com.example

import io.ktor.http.ContentType
import io.ktor.http.content.OutgoingContent
import io.ktor.http.content.TextContent
import io.ktor.http.withCharsetIfNeeded
import io.ktor.serialization.Configuration
import io.ktor.serialization.ContentConverter
import io.ktor.util.reflect.TypeInfo
import io.ktor.utils.io.ByteReadChannel
import io.ktor.utils.io.charsets.Charset
import io.ktor.utils.io.jvm.javaio.toInputStream
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext


data class CvtAResult(val data: String)

class CvtA : ContentConverter {
    override suspend fun deserialize(charset: Charset, typeInfo: TypeInfo, content: ByteReadChannel): Any? {
        return if (typeInfo.type == CvtAResult::class) {
            withContext(Dispatchers.IO) {
                val reader = content.toInputStream().reader(charset)
                CvtAResult(reader.readText())
            }
        } else {
            null
        }
    }

    override suspend fun serializeNullable(
        contentType: ContentType,
        charset: Charset,
        typeInfo: TypeInfo,
        value: Any?
    ): OutgoingContent? {
        return if (value != null && typeInfo.type == CvtAResult::class) {
            TextContent((value as CvtAResult).data, contentType.withCharsetIfNeeded(charset))
        } else {
            null
        }
    }
}

/**
 * Register a converter for org.JSON objects.
 */
fun Configuration.cvtA(
    contentType: ContentType = ContentType.Application.Json,
) {
    register(contentType, CvtA())
}

• CvtB:kt:

package com.example

import io.ktor.http.ContentType
import io.ktor.http.content.OutgoingContent
import io.ktor.http.content.TextContent
import io.ktor.http.withCharsetIfNeeded
import io.ktor.serialization.Configuration
import io.ktor.serialization.ContentConverter
import io.ktor.util.reflect.TypeInfo
import io.ktor.utils.io.ByteReadChannel
import io.ktor.utils.io.charsets.Charset
import io.ktor.utils.io.jvm.javaio.toInputStream
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext


data class CvtBResult(val data: String)

class CvtB : ContentConverter {
    override suspend fun deserialize(charset: Charset, typeInfo: TypeInfo, content: ByteReadChannel): Any? {
        return if (typeInfo.type == CvtBResult::class) {
            withContext(Dispatchers.IO) {
                val reader = content.toInputStream().reader(charset)
                CvtBResult(reader.readText())
            }
        } else {
            null
        }
    }

    override suspend fun serializeNullable(
        contentType: ContentType,
        charset: Charset,
        typeInfo: TypeInfo,
        value: Any?
    ): OutgoingContent? {
        return if (value != null && typeInfo.type == CvtBResult::class) {
            TextContent((value as CvtBResult).data, contentType.withCharsetIfNeeded(charset))
        } else {
            null
        }
    }
}

/**
 * Register a converter for org.JSON objects.
 */
fun Configuration.cvtB(
    contentType: ContentType = ContentType.Application.Json,
) {
    register(contentType, CvtB())
}

Test request can be made via curl:

curl -i http://localhost:8080/  -H "Content-Type: application/json" --data '{}'

I expected the following output (behaviour of Ktor 2.1.3):

HTTP/1.1 200 OK
Content-Length: 8
Content-Type: text/plain; charset=UTF-8

ECHO: {}

However, with Ktor 2.2.1 this is printed:

HTTP/1.1 415 Unsupported Media Type
Content-Length: 0

As for the parallel use of org.json and kotlinx.serialization: I've previously used org.json only and then introduced kotlinx.serialization for handling data with static schema. I've now noticed that kotlinx.serialization.json provides JsonElement which appears to be able to handle data without a fixed schema. I might therefore switch away from using org.json entirely and thus this issue wouldn't affect me anymore.

Example route:

get("/api/subtree/{...}") {
    ...
}

Expected:

• /api/subtree - do not match (according to KTOR-372 logic)
• /api/subtree/ - match
• /api/subtree/lvl1 - match
• /api/subtree/lvl1/ - match
• /api/subtree/lvl1/lvl2 - match
• /api/subtree/lvl1/lvl2/ - match

Current behavior:

• /api/subtree - match
• /api/subtree/ - do not match
• /api/subtree/lvl1 - match
• /api/subtree/lvl1/ - do not match
• /api/subtree/lvl1/lvl2 - match
• /api/subtree/lvl1/lvl2/ - do not match

I read about using two routes (with and without trailing '/' to cover both cases) or implementing redirect logic.
Those will work for static routes, but how to solve this for wildcard subtree route?

I would like to request the addition of HTTPS to the schema argument of allowHost in CORSConfig. Currently, when only HTTP is used, it works correctly in a local environment even without setting the schema argument. However, when deployed to an HTTPS development environment, the absence of a schema argument causes it not to function, leading to time-consuming troubleshooting.

https://api.ktor.io/ktor-server/ktor-server-plugins/ktor-server-cors/io.ktor.server.plugins.cors/-c-o-r-s-config/allow-host.html

Instances of io.ktor.http.RequestConnectionPoint are sometimes used in logs, for example here:

https://github.com/ktorio/ktor/blob/2.3.7/ktor-server/ktor-server-plugins/ktor-server-cors/jvmAndNix/src/io/ktor/server/plugins/cors/CORS.kt#L107

This produces logs like this this Netty for example:

12:13:39.734 TRACE - Respond forbidden /: origin doesn't match io.ktor.server.netty.http1.NettyConnectionPoint@7a5bf5c3

Such log does not help with debugging CORS, I don't know if there are more instances of origin used in logs.

Hello, I have a web server that is built using Ktor, and is deployed on Heroku, In this server I configured WebSockets and created the following route:

fun Application.configureSockets() {

    val json by inject<Json>()

    install(WebSockets) {
        pingPeriod = Duration.ofSeconds(15)
        timeout = Duration.ofSeconds(15)
        maxFrameSize = Long.MAX_VALUE
        masking = false
        contentConverter = KotlinxWebsocketSerializationConverter(json)
    }
}

fun Route.getEnrolledStudentSocketRoute() {

    val services by inject<ClassServices>()

    webSocket(application.href(EndPoint.Socket.Class.EnrolledStudents())) {

        val classId = call.parameters["classId"] ?: return@webSocket close(CloseReason(CloseReason.Codes.CANNOT_ACCEPT, "class id is missing"))

        services.getEnrolledStudents(classId).collectLatest { students ->
            sendSerialized(EnrolledStudentsDto(classId, students.map(Student::asDto)))
        }
    }
}

From the client side, I have an Android application with Ktor Client, which is configured to use websockets, I have this request:

internal fun HttpClientConfig<*>.configureWebSockets(json: Json) {
    WebSockets {
        contentConverter = KotlinxWebsocketSerializationConverter(json)
    }
}

override suspend fun getEnrolledStudents(classId: String) = channelFlow {

  val url = client.href(TeacherStudentPackagesApi.WebSocket.EnrolledStudents(classId))

  client.wss(url) {
      incoming.consumeEach {
          this@channelFlow.send(receiveDeserialized())
      }
  }
}

However, when I try to establish the connection, the server always throw this error:

java.lang.IllegalStateException: Unable to perform upgrade as it is not requested by the client: request should have Upgrade and Connection headers filled properly
	at io.ktor.server.cio.CIOApplicationResponse.respondOutgoingContent(CIOApplicationResponse.kt:114)
	at io.ktor.server.engine.BaseApplicationResponse$Companion$setupSendPipeline$1.invokeSuspend(BaseApplicationResponse.kt:317)
	at io.ktor.server.engine.BaseApplicationResponse$Companion$setupSendPipeline$1.invoke(BaseApplicationResponse.kt)
	at io.ktor.server.engine.BaseApplicationResponse$Companion$setupSendPipeline$1.invoke(BaseApplicationResponse.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(SuspendFunctionGun.kt:88)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invokeSuspend(DefaultTransform.kt:29)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
	at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.websocket.RoutingKt.respondWebSocketRaw(Routing.kt:293)
	at io.ktor.server.websocket.RoutingKt.access$respondWebSocketRaw(Routing.kt:1)
	at io.ktor.server.websocket.RoutingKt$webSocketRaw$2$1$1$1.invokeSuspend(Routing.kt:105)
	at io.ktor.server.websocket.RoutingKt$webSocketRaw$2$1$1$1.invoke(Routing.kt)
	at io.ktor.server.websocket.RoutingKt$webSocketRaw$2$1$1$1.invoke(Routing.kt)
	at io.ktor.server.routing.Route$buildPipeline$1$1.invokeSuspend(Route.kt:116)
	at io.ktor.server.routing.Route$buildPipeline$1$1.invoke(Route.kt)
	at io.ktor.server.routing.Route$buildPipeline$1$1.invoke(Route.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
	at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invoke(Pipeline.kt)
	at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invoke(Pipeline.kt)
	at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
	at io.ktor.server.routing.Routing.executeResult(Routing.kt:190)
	at io.ktor.server.routing.Routing.interceptor(Routing.kt:64)
	at io.ktor.server.routing.Routing$Plugin$install$1.invokeSuspend(Routing.kt:140)
	at io.ktor.server.routing.Routing$Plugin$install$1.invoke(Routing.kt)
	at io.ktor.server.routing.Routing$Plugin$install$1.invoke(Routing.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invokeSuspend(BaseApplicationEngine.kt:124)
	at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invoke(BaseApplicationEngine.kt)
	at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invoke(BaseApplicationEngine.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.server.application.hooks.CallFailed$install$1$1.invokeSuspend(CommonHooks.kt:45)
  at io.ktor.server.application.hooks.CallFailed$install$1$1.invoke(CommonHooks.kt)
	at io.ktor.server.application.hooks.CallFailed$install$1$1.invoke(CommonHooks.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:78)
	at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:264)
	at io.ktor.server.application.hooks.CallFailed$install$1.invokeSuspend(CommonHooks.kt:44)
	at io.ktor.server.application.hooks.CallFailed$install$1.invoke(CommonHooks.kt)
	at io.ktor.server.application.hooks.CallFailed$install$1.invoke(CommonHooks.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
	at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invokeSuspend(DefaultEnginePipeline.kt:123)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invoke(DefaultEnginePipeline.kt)
	at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invoke(DefaultEnginePipeline.kt)
	at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
	at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
	at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
	at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
	at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
	at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
	at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
	at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
	at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2.invokeSuspend(CIOApplicationEngine.kt:239)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
	at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)

The request log from the client side:

REQUEST: wss://<BASE_URL>/ws/class/students?classId=default
METHOD: HttpMethod(value=GET)
COMMON HEADERS
-> Accept: application/json
-> Accept-Charset: UTF-8
-> Content-Type: application/json
CONTENT HEADERS
-> Connection: upgrade
-> Sec-WebSocket-Key: YTA4OWRkNDQ1YjU0YmVmYQ==
-> Sec-WebSocket-Version: 13
-> Upgrade: websocket
BODY Content-Type: null
BODY START

BODY END

That is weird, other clients such as postman can establish a successful connection, but the Android Ktor client does not. I am using CIO engine on both server and client, I have tried other JVM engines on my client, but all didn't work. From the above request log, found that `Upgrade` and `Connection` headers are properly filled, so I can't understand where is the problem exactly, It seems like a bug for me, I also tried some 2.3.x older versions, the error still arises.

Current State

Currently, the entry point of the docs is the 'Welcome' page. For a starting page, nothing really stands out apart from the Ktor logo. The 'featured topics' look like second navigation where a summary is provided for each topic on mouse hover. The summaries are not consistent in style and in some cases show random content (e.g. 'Custom plugins' shows 'code example: custom-plugin').

Proposed Solution

Restructure the page using cards and groups supplied by Writerside's starting pages. Along with this, the following changes can be made:

• change the title to "Ktor Documentation" (if keeping the logo, make it smaller)
• keep the introduction, but refine it to add more clarity and flow
• below the introduction, show buttons leading to the Getting Started tutorials, such as:{width=70%}
• the rest of the topics can be presented as grouped links with tabs for Client and Server. For example:
  • "First steps" to display links for the topics under "Configuring a server" and "Setting up a client" or core concepts, such as "Plugins"

If you try to use a gcr.io image as a base image for containerization, you will get the following error:

FAILURE: Build failed with an exception.

* What went wrong:

Execution failed for task ':jibDockerBuild'.

> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: Tried to pull image manifest for gcr.io/distroless/java17-debian11:debug but failed because: Manifest with tag 'debug' has media type 'application/vnd.oci.image.index.v1+json', but client accepts 'application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.v2+json,application/vnd.docker.distribution.manifest.v1+json,application/vnd.docker.distribution.manifest.list.v2+json'.

Adding the latest version of jib plugin explicitly solves the problem.

How to reproduce:

• generate fresh Ktor project at https://start.ktor.io/ (Ktor 2.3.4 as for now)
• try to ./gradlew -Djib.from.image='gcr.io/distroless/java17-debian11:debug' jibDockerBuild and you will get the above error

Workaround:

Just add jib plugin explicitly.

plugins {
    kotlin("jvm") version "1.9.10"
    id("io.ktor.plugin") version "2.3.4"
    id("com.google.cloud.tools.jib") version "3.3.2" // <<------------
}

After bumping ktor client version from 2.3.3 to 2.3.4 we are getting a "Optional(io.ktor.utils.io.errors.EOFException: Premature end of stream: expected 467 bytes)) error" on iOS (works fine on Android) when sending a form with binary data.

The code simplified looks like this (image is a byte array):

val formData = formData {
    append(
        key = "image",
        value = image,
        headers = Headers.build {
            append(HttpHeaders.ContentType, "image/jpeg")
            append(HttpHeaders.ContentDisposition, "filename=image.jpeg")
        },
    )
}

return httpClient.submitFormWithBinaryData(url, formData) {}.body()

We didn't change anything on our side and downgrading to 2.3.3 fixes the issue, so it seems to be related to the update.

In execute() method in AndroidClientEngine class, following line makes it cannot handle content length that exceeds integer range.

contentLength?.let { setFixedLengthStreamingMode(it.toInt()) } ?: setChunkedStreamingMode(0)

Since there is setFixedLengthStreamingMode(long), removing toInt() call may fix the issue.

I guess the problem is caused by DarwinSession.close:

    override fun close() {
        if (!closed.compareAndSet(false, true)) return
        session.finishTasksAndInvalidate()
    }

In Apple's document,

This method returns immediately without waiting for tasks to finish. Once a session is invalidated, new tasks cannot be created in the session, but existing tasks continue until completion . After the last task finishes and the session makes the last delegate call related to those tasks, the session calls the URLSession:didBecomeInvalidWithError: method on its delegate, then breaks references to the delegate and callback objects. After invalidation, session objects cannot be reused.

To cancel all outstanding tasks, call invalidateAndCancel instead.

So, in close function, invoking invalidateAndCancel is better than finishTasksAndInvalidate

Many ktor client plugins contain their own LOGGER, e.g

internal val LOGGER = KtorSimpleLogger("io.ktor.client.plugins.websocket.WebSockets")

Which can't be configured.

Implementation of KtorSimpleLoggerJs.kt

@Suppress("FunctionName")
public actual fun KtorSimpleLogger(name: String): Logger = object : Logger {
    override fun error(message: String) {
        console.error(message)
    }

    override fun error(message: String, cause: Throwable) {
        console.error("$message, cause: $cause")
    }

    override fun warn(message: String) {
        console.warn(message)
    }

    override fun warn(message: String, cause: Throwable) {
        console.warn("$message, cause: $cause")
    }

    override fun info(message: String) {
        console.info(message)
    }

    override fun info(message: String, cause: Throwable) {
        console.info("$message, cause: $cause")
    }

    override fun debug(message: String) {
        console.debug("DEBUG: $message")
    }

    override fun debug(message: String, cause: Throwable) {
        console.debug("DEBUG: $message, cause: $cause")
    }

    override fun trace(message: String) {
        console.debug("TRACE: $message")
    }

    override fun trace(message: String, cause: Throwable) {
        console.debug("TRACE: $message, cause: $cause")
    }
}

It floods logs with unnecessary and useless messages on Node as console.debug is just an alias for console.log

TRACE: Sending WebSocket request [object Object]

This logger should be configurable with Logging plugin.

The following code fails with "Server sent a subprotocol but none was requested":

    val ktorClient = HttpClient(Js) {
      install(WebSockets)
    }

    val url = Url("wss://apollo-fullstack-tutorial.herokuapp.com/graphql")

    val connection = ktorClient.request<DefaultClientWebSocketSession>(url) {
      headers {
        append("Sec-WebSocket-Protocol", "graphql-ws")
      }
    }

This seems to happen because protocols is never passed to the WebSocket constructor (here)

Description:

When using the Ktor HTTP client library to download files, an issue has been observed where the client fails to recover after an interrupted download caused by a loss of internet connection. This behavior is particularly evident when the provided code snippet is used to perform the download.

Steps to Reproduce:

1. Establish a working internet connection.
2. Execute the provided code snippet for making an HTTP request to download a file using the Ktor HTTP client.
3. During the file download process, intentionally disrupt the internet connection.
4. Allow the client to handle the connection loss and observe its behavior.

Expected Behavior: Upon detecting a loss of connectivity during the file download process, the Ktor HTTP client should throw an appropriate exception (e.g., a network-related exception) to indicate the connection failure. Subsequent attempts to make new HTTP requests using the client, after restoring the internet connection, should work as expected. The client should no longer be stuck in an unrecoverable state and should be capable of handling further requests without hindrance.

Actual Behavior: Currently, when the internet connection is disrupted during the file download process, the Ktor HTTP client appears to enter an unrecoverable state. Subsequent attempts to use the client to make additional HTTP requests (even after the internet connection is restored) fail to complete successfully. The client seems to be stuck in a state where it cannot recover from the interrupted download.

Code Snippet:

try {
    val httpResponse: HttpResponse = httpClient.get(url) {
        onDownload { bytesSentTotal, contentLength ->
            // Progress tracking logic can be added here if needed 
        }
    }
    val responseBody: ByteArray = httpResponse.body()
    file.writeBytes (responseBody)
} catch (e: Exception) {
    // Exception handling logic can be added here
}

Exception Stack Trace:

java.net.SocketTimeoutException: timeout at com.android.okhttp.okio.Okio$3.newTimeoutException(Okio.java:214) at com.android.okhttp.okio.AsyncTimeout.exit(AsyncTimeout.java:263) at com.android.okhttp.okio.AsyncTimeout$2.read(AsyncTimeout.java:217) at com.android.okhttp.okio.RealBufferedSource.read(RealBufferedSource.java:51) at com.android.okhttp.internal.http.Http1xStream$FixedLengthSource.read(Http1xStream.java:395) at com.android.okhttp.okio.RealBufferedSource$1.read(RealBufferedSource.java:372) at java.io.BufferedInputStream.fill(BufferedInputStream.java:248) at java.io.BufferedInputStream.read1(BufferedInputStream.java:288) at java.io.BufferedInputStream.read(BufferedInputStream.java:347) at io.ktor.utils.io.jvm.javaio.ReadingKt$toByteReadChannel$1.invokeSuspend(Reading.kt:55) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684) Caused by: java.net.SocketException: Socket closed at java.net.SocketInputStream.read(SocketInputStream.java:188) at java.net.SocketInputStream.read(SocketInputStream.java:143) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.readFromSocket(ConscryptEngineSocket.java:945) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.processDataFromSocket(ConscryptEngineSocket.java:909) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.readUntilDataAvailable(ConscryptEngineSocket.java:824) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.read(ConscryptEngineSocket.java:797) at com.android.okhttp.okio.Okio$2.read(Okio.java:138) at com.android.okhttp.okio.AsyncTimeout$2.read(AsyncTimeout.java:213) at com.android.okhttp.okio.RealBufferedSource.read(RealBufferedSource.java:51) at com.android.okhttp.internal.http.Http1xStream$FixedLengthSource.read(Http1xStream.java:395) at com.android.okhttp.okio.RealBufferedSource$1.read(RealBufferedSource.java:372) at java.io.BufferedInputStream.fill(BufferedInputStream.java:248) at java.io.BufferedInputStream.read1(BufferedInputStream.java:288) at java.io.BufferedInputStream.read(BufferedInputStream.java:347) at io.ktor.utils.io.jvm.javaio.ReadingKt$toByteReadChannel$1.invokeSuspend(Reading.kt:55) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)

Note:

The preceding stack trace remains consistent for both the unsuccessful download request and the subsequent requests that follow it.

Environment:

• Ktor version: [2.3.3]
• HttpClientEngine: Android
• Operating System: [Android 11]

Hello,

I have encountered an issue while trying to establish a WebSocket connection using Ktor. It appears that the 'Connection: upgrade' header sent by Ktor has 'upgrade' in lowercase, which is causing the WebSocket connection to fail.

In my application scenario, I tried CIO, Java, and OkHttp engines and captured packets for analysis. Only the OkHttp engine used the header with capital letters when establishing a connection, that is, "Connection: Upgrade", and then the connection was successful, while the other both engines use lowercase "upgrade", which causes the connection to fail.

By reading the code, I found that Ktor used lowercase "upgrade" here. I believe adjusting the capitalization to 'Connection: Upgrade' may resolve the WebSocket connection issue. I hope this information is helpful and look forward to hearing back on whether this could be addressed in a future release of Ktor.

Thank you.

It appears that sending a frame with an empty body (e.g. a Frame.Text("")) crashes an internal coroutine that fails incoming channel collections.

The root cause is the following:

Caused by: kotlin.ArrayIndexOutOfBoundsException
    at 0   ???                                 7ff7ae9c8d62       kfun:kotlin.Throwable#<init>(){} + 98 
    at 1   ???                                 7ff7ae9c2f7f       kfun:kotlin.Exception#<init>(){} + 79 
    at 2   ???                                 7ff7ae9c31ef       kfun:kotlin.RuntimeException#<init>(){} + 79 
    at 3   ???                                 7ff7ae9c337f       kfun:kotlin.IndexOutOfBoundsException#<init>(){} + 79 
    at 4   ???                                 7ff7ae9c3a4f       kfun:kotlin.ArrayIndexOutOfBoundsException#<init>(){} + 79 
    at 5   ???                                 7ff7ae9f1028       ThrowArrayIndexOutOfBoundsException + 120 
    at 6   ???                                 7ff7aed892a7       Kotlin_Arrays_getByteArrayAddressOfElement + 23 
    at 7   ???                                 7ff7ae9b8ed1       kfun:kotlinx.cinterop#addressOf__at__kotlinx.cinterop.Pinned<kotlin.ByteArray>(kotlin.Int){}kotlinx.cinterop.CPointer<kotlinx.cinterop.ByteVarOf<kotlin.Byte>> + 177 
    at 8   ???                                 7ff7aed26b9d       kfun:io.ktor.client.engine.winhttp.internal.WinHttpWebSocket.sendFrame#internal + 2877 
    at 9   ???                                 7ff7aed255f4       kfun:io.ktor.client.engine.winhttp.internal.WinHttpWebSocket.$sendNextFrameCOROUTINE$0.invokeSuspend#internal + 2580 
    at 10  ???                                 7ff7ae9ce785       kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 1029

It turns out the WinHttpWebSocket tries to use buffer.addressOf(0) even when the body (buffer) is empty:
https://github.com/ktorio/ktor/blob/b34415dafca82090e408761788a9b7ffb24788d7/ktor-client/ktor-client-winhttp/windows/src/io/ktor/client/engine/winhttp/internal/WinHttpWebSocket.kt#L201

I think the same issue is probably true for receiving frames with empty bodies:
https://github.com/ktorio/ktor/blob/b34415dafca82090e408761788a9b7ffb24788d7/ktor-client/ktor-client-winhttp/windows/src/io/ktor/client/engine/winhttp/internal/WinHttpWebSocket.kt#L109

Note that it might be hard to notice due to https://youtrack.jetbrains.com/issue/KT-62794 (we need to use printStackTrace() to see the cause).

Reproducer:

fun main() {
    runBlocking {
        val http = HttpClient(WinHttp) {
            install(WebSockets)
        }
        val session = http.webSocketSession("wss://ws.postman-echo.com/raw")
        try {
            session.send(Frame.Text(""))
            
            // This fails with ArrayIndexOutOfBoundsException, not because of the received echoed frame,
            // but because of the failed send causing the incoming channel to fail (see sendFrame in the stacktrace)
            session.incoming.receive()
        } finally {
            session.close()
        }
    }
}

Affected topics:

• https://helpserver.labs.jb.gg/help/ktor/2.0.0/creating-http-apis.html#dependencies
• https://helpserver.labs.jb.gg/help/ktor/2.0.0/creating-static-website.html#dependencies
• https://helpserver.labs.jb.gg/help/ktor/2.0.0/creating-web-socket-chat.html#dependencies

I think We have the same bug
{width=70%}

After updating ktor plugin (for gradle) to version 2.3.6, we also updated jackson-core-2.15.2 . But it has some problems with gradle 7.5.1 .
With the ktor plugin 2.3.5, it works fine

Because of this, the newly created project does not build in ide

Description:
When using embeddedCIOServer and geting all headers with call.response.headers.allValues() returned values are not as expected (examples below). When switching to another engine(e.g. Netty) returned values are as expected.

Describe the bug:

Function allValues() uses abstract function getEngineHeaderValues(name: String) from io.ktor.server.cio.CIOApplicationResponse and its implementation in cio-server is pretty confusing and in my testing wrong. I'm setting custom header for a route response like this:

call.response.headers.append("Test-Header", "Test-Value")

when getting header values the output is

{Test-Header=[Test-Value, Test-Value, Test-Value]}

When adding additional header with the same name:

call.response.headers.append("Test-Header", "Test-Value2")

the output is:

{Test-Header=[Test-Value, Test-Value, Test-Value, Test-Value2, Test-Value, Test-Value, Test-Value, Test-Value2]}

Output when using netty server:

{Test-Header=[Test-Value]}

{Test-Header=[Test-Value, Test-Value2, Test-Value, Test-Value2]}

*Second output is also correct since there are 2 headers with 2 values but with the same key

To reproduce:
Create simple CIO and Netty servers, add headers on the route and get response headers via allValues() function.

Or clone the project https://github.com/Theanko1412/ktor-issue-demo

Additional information:

After looking at git history this function was changed in commit from 2017, I mocked old behavior of this function with allValuesFixed() in my demo project and it is working as expected.
The response is having correct values, problem is only in retrieving it with allValues()

Proposed solution:
Revert function to its previous state.

If someone can provide some more information why was this changed and how was current implementation meant to be used I would appreciate it. Currently don't see any other way to correctly retrieve all headers from a response other than this.

Issue seems to be pretty old but couldn't find any old issues about this, if this is duplicate I would appreciate if you can point me to the solution and reasons why am I getting triple header value.

Hello,

since version 2.3.1 (I suspect it came with the change https://youtrack.jetbrains.com/issue/KTOR-5797, but that's just a guess), it is not possible anymore to access a YAML configuration value that is assigned an explicit null value via the propertyOrNull method.

Example:

ktor:
    application:
        modules:
            - com.example.ApplicationKt.module
    deployment:
        port: 8080
    sample:
        prop1: null
#        pro2: null

package com.example

import io.ktor.server.application.*
import io.ktor.server.response.*
import io.ktor.server.routing.*

fun main(args: Array<String>) {
    io.ktor.server.netty.EngineMain.main(args)
}

fun Application.module() {
    routing {
        get("/") {
            val config = call.application.environment.config

            call.application.log.info("prop1: ${config.propertyOrNull("ktor.sample.prop1") == null}")
            call.application.log.info("prop2: ${config.propertyOrNull("ktor.sample.prop2") == null}")

            call.respondText("OK")
        }
    }
}

With up to version 2.3.0, it runs fine, but starting at 2.3.1, it fails as follows:

ApplicationConfigurationException: Expected primitive or list at path ktor.sample.prop1, but was class net.mamoe.yamlkt.YamlNull

I suspect that's the case because, in the YamlConfig#propertyOrNull implementation the line val value = yaml[parts.last()] ?: return null does not work as intended because the evaluation of the expression yaml[parts.last()] yields an instance of YamlNull instead of null itself. Therefore, the early exit with the evlis operator ?: return null does not fire. But again, that's just a suspicion of mine.

I am not sure if this is entirely intentional or if it indicates a bug. If it does, I would be glad if you could fix it. My temporary workaround is commenting out configuration variables whose values should be null instead of explicitly assigning null because then it behaves as previously.

I have attached the repro example from above as zip.

BaseApplicationEngine launches job on application coroutine context which awaits resolvedConnectors to complete. They are completed successfully on start with NettyApplicationEngine but never completes with TestApplicationEngine.

In tests, we'd like to await all the async tasks that were started with code:

withTimeoutOrNull(timeout) { application.coroutineContext.job.children.forEach { scope -> scope.join() } } ?: throw TimeoutException("Timed out waiting ($timeout) for child coroutines to finish")

but the resolvedConnectors job is still awaiting resolved connectors.

As a workaround, the first thing we do (after engine initialization) is
engine.application.coroutineContext.job.children.firstOrNull()?.cancelAndJoin()

but this is hacky a little bit.

Our project uses Kotlin 1.7. It's now unable to benefit from this "bugfix release", due to binary incompatibility.

e.g.

lass 'io.ktor.http.HttpHeaders' was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.9.0, expected version is 1.7.1.

See attached fatal error log that was generated because IDEA failed to allocate a small amount of native memory (via malloc()) on an M1 Max machine with 64G physical memory after 2 days of uptime. In the log, there are 300+ threads like this (see below) that alone eat up 2.5G+ memory just for their stacks. This does not look right and could be (but not definitely is) a reason for OOME:

  0x000000014420b400 JavaThread "HttpClient-22-SelectorManager" daemon [_thread_in_native, id=64651, stack(0x00000002cae50000,0x00000002cb053000)]
  0x0000000136207600 JavaThread "ktor-client-java-2" daemon [_thread_blocked, id=58891, stack(0x00000002cb05c000,0x00000002cb25f000)]
  0x0000000142868e00 JavaThread "ktor-client-java-3" daemon [_thread_blocked, id=40967, stack(0x000000029d0a0000,0x000000029d2a3000)]
  0x000000032aa29000 JavaThread "Netty Builtin Server 3" daemon [_thread_in_native, id=47419, stack(0x00000002ceffc000,0x00000002cf1ff000)]
  0x0000000136e5c800 JavaThread "ktor-client-java-0" daemon [_thread_blocked, id=51255, stack(0x000000033f490000,0x000000033f693000)]
  0x000000030bf58400 JavaThread "HttpClient-35-SelectorManager" daemon [_thread_in_native, id=165147, stack(0x000000033f69c000,0x000000033f89f000)]
  0x000000030bf58a00 JavaThread "ktor-client-java-1" daemon [_thread_blocked, id=83379, stack(0x00000003595c0000,0x00000003597c3000)]
  0x0000000147f23400 JavaThread "ktor-client-java-2" daemon [_thread_blocked, id=66847, stack(0x0000000364d18000,0x0000000364f1b000)]
  0x0000000147e5e000 JavaThread "ktor-client-java-3" daemon [_thread_blocked, id=27415, stack(0x0000000365410000,0x0000000365613000)]

When upgrading from Apache to Apache5 we noticed bottlenecking on our HTTP requests. Digging through the Apache engine code we noticed that https://github.com/ktorio/ktor/blob/0d63c4d8d987098071527125e2bb4c11da9b2aa3/ktor-client/ktor-client-apache5/jvm/src/io/ktor/client/engine/apache5/Apache5Engine.kt#L94-L95 got typod. One of those is supposed to be setMaxConnPerRoute.

If I set a pingInterval bigger than 0 on the DarwinClientEngine websocket it will invariably timeout because it will reject all received pongs as invalid.

[TRACE] (io.ktor.websocket.WebSocket): WebSocket Pinger: sending ping frame
[TRACE] (io.ktor.websocket.WebSocket): WebSocketSession(StandaloneCoroutine{Active}@f84385f0) receiving frame Frame PONG (fin=true, buffer len = 0)
[TRACE] (io.ktor.websocket.WebSocket): WebSocket Pinger: received invalid pong frame Frame PONG (fin=true, buffer len = 0), continue waiting

So I'm taking a look at PingPong.kt to see what validation is done

// wait for valid pong message
while (true) {
    val msg = channel.receive()
    if (String(msg.data, charset = Charsets.ISO_8859_1) == pingMessage) {
        LOGGER.trace("WebSocket Pinger: received valid pong frame $msg")
        break
    }

    LOGGER.trace("WebSocket Pinger: received invalid pong frame $msg, continue waiting")
}

So it expects the received pong to have the same payload as the outgoing ping. Let's take look at the DarwinWebsocketSession.kt and see what it does with the payload.

FrameType.PING -> {
    task.sendPingWithPongReceiveHandler { error ->
        if (error != null) {
            cancel("Error receiving pong", DarwinHttpRequestException(error))
            return@sendPingWithPongReceiveHandler
        }
        _incoming.trySend(Frame.Pong(ByteReadPacket.Empty))
    }
}

Here we can see that it always forwards an empty pong since the Darwin API doesn't expose the payload when sending and receiving ping & pongs.

Since Darwin APIs doesn't support payloads when sending pings maybe there should be an option to configure the pinger without any validation on the payload? The way it's implemented now the pinger just doesn't work at all on Darwin platforms.

Locally there is a version incompatibility with '2.3.5' when running the code sample. Furthermore, refer to the thread on Slack to verify the tutorial, specifically the setup for iOS.

https://jbs.zendesk.com/agent/tickets/5662082

To reproduce:

Create maven project

Try to run Ktor Application

Expected:

Application started successfully

Actual:

Error:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

You can fix it by adding dependency:

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>${slf4j.version}</version>
</dependency>

The max JRE version in ktor docker plugin is JRE_17, but JRE_19 and JRE_20 are already out.
If possible, a suggested approach is to use a string type for that field.

When device is in Airplane Mode and a request with MultiPartFormDataContent is started, the app gets stuck in an infinite loop (in OutgoingContent.toDataOrStream() at line 52).

Sample request:

        httpClient.post("...") {
            setBody(
                MultiPartFormDataContent(
                    formData {
                        append(
                            "file",
                            content,
                            Headers.build {
                                append(HttpHeaders.ContentType, "application/pdf")
                                append(HttpHeaders.ContentDisposition, "filename=\"file.pdf\"")
                            }
                        )
                    }
                )
            )
        }

The problem in OutgoingContent.toDataOrStream() is that outputStream.write in my case returns -1 and thus the condition of the While loop (offset < read) never becomes false.

Some people new to Ktor cannot understand the cause of the NoTransformationFoundException and how to fix the problem.

Ktor Version and Engine Used (client or server and name)
2.2.3, OkHttp

Describe the bug
When sending Cookie header with multiple name-value pairs, they are separated with a comma instead of a semicolon.
See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cookie#directives

To Reproduce

val response = client.post("http://localhost") {
    cookie("firstCookie", "first")
}.body<HttpResponse>()

with the DefaultRequest plugin set up like this :

defaultRequest {
    cookie("secondCookie", "second")
}

generates the following request :

curl -X POST "http://localhost"  -H "Cookie:firstCookie=first,secondCookie=second"

Expected behavior
The Cookie header should be:

Cookie: firstCookie=first; secondCookie=second

Remarks

• The function cookie handles multiple name-value pairs correctly
• The issue happens when adding cookies as headers "manually", with header(HttpHeaders.Cookie, "firstCookie=first")
• It also happens when using cookie in both the request builder and the defaultRequest builder.
• This way, the Cookie header is created by the function mergeHeaders which uses a comma for every headers.
• It has been changed to fix this issue about the Accept header

To reproduce use the following code for a client and server:

val client = HttpClient(CIO) {
    install(Auth) {
        digest {
            algorithmName = "SHA-256"
            credentials {
                DigestAuthCredentials(username = "jetbrains", password = "foobar")
            }
            realm = "Access to the '/' path"
        }
    }
}

val r = client.get<String>("http://localhost:9090/secret")
println(r)

fun getMd5Digest(str: String): ByteArray = MessageDigest.getInstance("MD5").digest(str.toByteArray(UTF_8))

val myRealm = "Access to the '/' path"
val userTable: Map<String, ByteArray> = mapOf(
    "jetbrains" to getMd5Digest("jetbrains:$myRealm:foobar"),
    "admin" to getMd5Digest("admin:$myRealm:password")
)

embeddedServer(Netty, port = 9090) {
    install(CORS) {
        anyHost()
        allowHeaders { true }
        exposeHeader(HttpHeaders.WWWAuthenticate)
        HttpMethod.DefaultMethods.forEach { method(it) }
    }

    install(Authentication) {
        digest("auth-digest") {
            algorithmName = "SHA-256"
            realm = myRealm
            digestProvider { userName, _ ->
                userTable[userName]
            }
        }
    }

    routing {
        authenticate("auth-digest") {
            get("/secret") {
                call.respondText { "Secret" }
            }
        }
    }
}.start()

As a result, the 401 status code is return instead of expected 200. It works as expected only if the default MD5 algorithm is used.

Several improvements can be made to the tutorials for creating a websocket chat:

1. Server

• the dependencies generated by the plugin look different and do not include the ch.qos.logback:logback-classic
• in source code the routing function is missing from the first appearance of configureSockets
• feature requests need formatting
• remove the use of plural formatted keywords (ex. Frames)

2. Client

• screenshots are missing in Create a new project

Hi

It seems that the io.ktor.plugin.features.JreVersion no longer exists. So this is not compiling:

ktor {
    docker {
        jreVersion.set(io.ktor.plugin.features.JreVersion.JRE_17)....

Create a dummy new project using File | New | Project | Ktor with default settings.
Wait for the project to be imported.
It won't import and fail with error
e: /Users/dmitrii.naumenko/Desktop/dev/ktor-sample/build.gradle.kts:28:42: Unresolved reference: slf4j_version

I don't see any slf4j_version in any configuration file in the newly created project.
It's quite a poor Kotlin newcomer experience, especially for an Ultimate feature

---

IntelliJ IDEA 2023.2.2 (Ultimate Edition)
Build #IU-232.9921.88, built on September 30, 2023
Licensed to JetBrains Team / Dmitrii Naumenko
You have a perpetual fallback license for this version.
Subscription is active until December 28, 2026.
Runtime version: 17.0.8+7-b1000.22 aarch64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 13.5.2
GC: G1 Young Generation, G1 Old Generation
Memory: 8000M
Cores: 12
Metal Rendering is ON
Registry:
debugger.auto.attach.from.console=true
ide.experimental.ui.show.resume=true
ide.smart.update=true
ide.mac.file.chooser.native=false
ide.experimental.ui=true
ea.indicator.blinking.timeout=0
shared.indexes.jdk.download=false
scala.erase.compiler.process.jdk.once=false
shared.indexes.maven.download=false
shared.indexes.download.auto.consent=true
scala.xray.mode=true

Non-Bundled Plugins:
de.netnexus.camelcaseplugin (3.0.12)
org.nik.presentation-assistant (1.0.10)
org.jetbrains.idea.grammar (2022.3.2)
jetbrains.team.auth (232.9921.32)
com.intellij.sisyphus (232.9921.31)
com.jetbrains.intellij.code.search.polaris (0.232.900.117)
io.intellij-sdk-thread-access (2.2.1)
com.jetbrains.idea.safepush (232.9921.32)
com.intellij.ml.llm (232.9943)
com.jetbrains.intellij.api.watcher (232.9921.31)
Pythonid (232.9921.47)
org.intellij.scala (2023.2.843)
com.intellij.git.instant (1.3.7)

Kotlin: 232-1.9.0-IJ9921.88

MicrometerMetricsConfig always creates LoggingMeterRegistry() by default, later if register is reset to something else, LoggingMeterRegister won't get closed automatically.

pluginOrNull(MicrometerMetrics)
        ?: install(MicrometerMetrics) {
            registry = AdminModule.meterRegistry
            timers { call, _ -> jsonRpcMetricTags(call).forEach { (k, v) -> tag(k, v) } }
        }

the coroutine created by LoggingMeterRegister leaks. a workaround is to add registry.close before re-assign. This is counter intuitive, most user won't do it.

There are several options,

1. make the default registery a no-op, this may affect open box experience, but it can document it better,
2. use a setter for register, in setter, it should always close previous resource.

To reproduce run the following test:

@Test
fun test() = testApplication {
    environment {
        config = MapApplicationConfig()
    }
    routing {
        get {
            call.respondText("Hello", contentType = ContentType.Application.Json)
        }
    }

    val r = client.get("/")
    assertEquals("application/json", r.contentType().toString())
}

There was an error when I used the AI Assistant.

• Error in UI

Unknown error. Try to repeat your request. If the issue persists, contact us.

• Error in idea.log

INFO - STDERR - Exception in thread "DefaultDispatcher-worker-5" java.lang.NullPointerException: {
INFO - STDERR -         sendAsync(http…xt)).await().body()
INFO - STDERR -     } must not be null
INFO - STDERR -     at io.ktor.client.engine.java.JavaHttpResponseKt.executeHttpRequest(JavaHttpResponse.kt:19)
INFO - STDERR -     at io.ktor.client.engine.java.JavaHttpResponseKt$executeHttpRequest$1.invokeSuspend(JavaHttpResponse.kt)
INFO - STDERR -     at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
INFO - STDERR -     at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
INFO - STDERR -     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
INFO - STDERR -     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
INFO - STDERR -     at java.base/java.lang.Thread.run(Thread.java:833)
INFO - STDERR -     Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [CoroutineName(com.intellij.ml.llm.core.chat.services.AIAssistantServiceProjectScope), StandaloneCoroutine{Cancelling}@ff251b2, Dispatchers.Default]

• IDEA Version.

IntelliJ IDEA 2023.2 (Ultimate Edition)
Build #IU-232.8660.185, built on July 26, 2023
Subscription is active until December 14, 2023.
Runtime version: 17.0.7+7-b1000.6 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Windows 10.0
GC: G1 Young Generation, G1 Old Generation
Memory: 8192M
Cores: 12
Registry:
    debugger.new.tool.window.layout=true
    ide.experimental.ui=true
    editor.minimap.enabled=true

Non-Bundled Plugins:
    com.intellij.javafx (1.0.4)
    Batch Scripts Support (1.0.13)
    dev.meanmail.plugin.nginx-intellij-plugin (2022.1.1)
    com.intellij.nativeDebug (232.8660.142)
    org.jetbrains.idea.grammar (2022.3.2)
    com.intellij.guice (232.8660.142)
    com.jetbrains.jax.ws (232.8660.142)
    com.intellij.plugin.adernov.powershell (2.0.10)
    PlantUML integration (6.3.0-IJ2023.2)
    Pythonid (232.8660.185)
    org.exbin.deltahex.intellij (0.2.8.1)
    ru.adelf.idea.dotenv (2023.2)
    androidx.compose.plugins.idea (232.8660.142)
    com.intellij.ml.llm (232.9353)
    org.jetbrains.plugins.docker.gateway (232.8660.202)

Kotlin: 232-1.9.0-IJ8660.185

• AI Assistant version: 232.9353

• Note: In my company, I must use a proxy server with authentication to connect to the internet.

We're investigating the impact of iOS 17 & Xcode 15 on our software, which is partly developed using Kotlin Multiplatform. We currently use Ktor 1.6.8 in our common code.

With Xcode 15 one of our unit test fails, while it worked fine with Xcode 14, with this error:

Uncaught Kotlin exception: kotlin.IllegalArgumentException: Failed to open iconv for charset UTF-8 with error code 22

This is the Ktor related part of the stack trace:

Uncaught Kotlin exception: kotlin.IllegalArgumentException: Failed to open iconv for charset UTF-8 with error code 22
kfun:kotlin.Throwable#<init>(kotlin.String?){} + 88 (/Users/teamcity1/teamcity_work/c3a91df21e46e2c8/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Throwable.kt:24:37)
kfun:kotlin.Exception#<init>(kotlin.String?){} + 86 (/Users/teamcity1/teamcity_work/c3a91df21e46e2c8/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 86 (/Users/teamcity1/teamcity_work/c3a91df21e46e2c8/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
kfun:kotlin.IllegalArgumentException#<init>(kotlin.String?){} + 86 (/Users/teamcity1/teamcity_work/c3a91df21e46e2c8/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:59:44)
kfun:io.ktor.utils.io.charsets.checkErrors#internal + 836 (/Users/administrator/Documents/agent/work/49d4a482a8522285/ktor-io/posix/src/io/ktor/utils/io/charsets/CharsetNative.kt:54:15)
kfun:io.ktor.utils.io.charsets.CharsetImpl.<init>#internal + 782 (/Users/administrator/Documents/agent/work/49d4a482a8522285/ktor-io/posix/src/io/ktor/utils/io/charsets/CharsetNative.kt:27:9)
kfun:io.ktor.utils.io.charsets.Charsets#<init>(){} + 224 (/Users/administrator/Documents/agent/work/49d4a482a8522285/ktor-io/posix/src/io/ktor/utils/io/charsets/CharsetNative.kt:381:40)
InitSingletonStrict + 964 
kfun:io.ktor.http.cio.internals.$DefaultHttpMethods$lambda-1$FUNCTION_REFERENCE$16.$<bridge-BNNNN>invoke(-1:0;-1:1){}kotlin.Char#internal + 1548 
Kotlin_initRuntimeIfNeeded + 1572 
+[KotlinBase allocWithZone:] + 23
 

I am trying to build my project with linuxArm64 as target and get the following error:

Execution failed for task ':compileKotlinLinuxArm64'.
> Could not resolve all files for configuration ':linuxArm64CompileKlibraries'.
   > Could not resolve io.ktor:ktor-client-core-native:1.3.2.
     Required by:
         project :
      > Unable to find a matching variant of io.ktor:ktor-client-core-native:1.3.2:
          - Variant 'iosArm32-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'ios_arm32'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'iosArm64-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'ios_arm64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'iosX64-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'ios_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'js-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'js'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
          - Variant 'js-runtime' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attributes:
                  - Required org.gradle.usage 'kotlin-api' and found incompatible value 'kotlin-runtime'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'js'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
          - Variant 'jvm-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'jvm'.
              - Other attributes:
                  - Found org.gradle.libraryelements 'jar' but wasn't required.
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'java-api'.
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
          - Variant 'jvm-runtime' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'jvm'.
              - Other attributes:
                  - Found org.gradle.libraryelements 'jar' but wasn't required.
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'java-runtime'.
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
          - Variant 'linuxX64-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'linux_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'macosX64-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'macos_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'metadata-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'common'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
          - Variant 'mingwX64-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'mingw_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'tvosArm64-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'tvos_arm64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'tvosX64-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'tvos_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'watchosArm32-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'watchos_arm32'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'watchosArm64-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'watchos_arm64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'watchosX86-api' capability io.ktor:ktor-client-core-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'watchos_x86'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
   > Could not resolve io.ktor:ktor-client-curl:1.3.2.
     Required by:
         project :
      > Unable to find a matching variant of io.ktor:ktor-client-curl:1.3.2:
          - Variant 'linuxX64-api' capability io.ktor:ktor-client-curl:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'linux_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'macosX64-api' capability io.ktor:ktor-client-curl:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'macos_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'metadata-api' capability io.ktor:ktor-client-curl:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'common'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
          - Variant 'mingwX64-api' capability io.ktor:ktor-client-curl:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'mingw_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
   > Could not resolve io.ktor:ktor-client-logging-native:1.3.2.
     Required by:
         project :
      > Unable to find a matching variant of io.ktor:ktor-client-logging-native:1.3.2:
          - Variant 'iosArm32-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'ios_arm32'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'iosArm64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'ios_arm64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'iosX64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'ios_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'js-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'js'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
          - Variant 'js-runtime' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attributes:
                  - Required org.gradle.usage 'kotlin-api' and found incompatible value 'kotlin-runtime'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'js'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
          - Variant 'jvm-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'jvm'.
              - Other attributes:
                  - Found org.gradle.libraryelements 'jar' but wasn't required.
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'java-api'.
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
          - Variant 'jvm-runtime' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'jvm'.
              - Other attributes:
                  - Found org.gradle.libraryelements 'jar' but wasn't required.
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'java-runtime'.
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
          - Variant 'linuxX64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'linux_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'macosX64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'macos_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'metadata-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'common'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
          - Variant 'mingwX64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'mingw_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'tvosArm64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'tvos_arm64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'tvosX64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'tvos_x64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'watchosArm32-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'watchos_arm32'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'watchosArm64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'watchos_arm64'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
          - Variant 'watchosX86-api' capability io.ktor:ktor-client-logging-native:1.3.2:
              - Incompatible attribute:
                  - Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'watchos_x86'.
              - Other attributes:
                  - Found org.gradle.status 'release' but wasn't required.
                  - Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
                  - Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.

And linuxX64 works fine in the same project.

When HttpCache.Config.isShared is true, should be give priority to s-maxage over max-age.
This is why should be give priority to s-maxage over max-age.
https://httpwg.org/specs/rfc7234.html#rfc.section.5.2.2.9

But s-maxage is not used because isShared member is not passed for a parameter of cache storing function.

Hey hey, first time creating an issue. Hope I'm doing everything correctly :-)

I'm facing this problem, when I try to prepare a HttpClient like this:

val httpClient = HttpClient(CIO) {
    engine {
        endpoint.connectTimeout = 10000
    }
    defaultRequest {
        url(defaultUrl)
        cookie("first-cookie", "foo")
    }
}

val request = httpClient.preparePost("/some-route") {
    headers {
        contentType(ContentType.Application.Json)
    }
    cookie("second-cookie", "bar")
}

request.execute()

This will send a request with the following cookies in the header:

INFO io.ktor.client.HttpClient - REQUEST: https://domain/some-route
METHOD: HttpMethod(value=POST)
COMMON HEADERS
-> Accept: application/json
-> Accept-Charset: UTF-8
-> Cookie: first-cookie=foo; first-cookie=foo; second-cookie=bar
CONTENT HEADERS
-> Content-Length: 34
-> Content-Type: application/json
BODY Content-Type: application/json
BODY START
... some body ...
BODY END

Any ideas why this happens?

Doing it like this instead would work (removing the first cookie from the defaultRequest:

val httpClient = HttpClient(CIO) {
    engine {
        endpoint.connectTimeout = 10000
    }
    defaultRequest {
        url(defaultUrl)
    }
}

val request = httpClient.preparePost("/some-route") {
    headers {
        contentType(ContentType.Application.Json)
    }
    cookie("first-cookie", "foo")
    cookie("second-cookie", "bar")
}

request.execute()

Thanks for your help already!

I am using ktor on top of the Spring Boot. I am using io.ktor:ktor-server-servlet

However, it does not support yaml configuratuin. Ktor has ktor-server-config-yaml dependency, but it does not work with Ktor Servlet.

After some investigation, I found that ktor does not support yaml configuration.

It will be nice if somebody implement this feature.

I want to configure ktor and spring boot in same application.yaml

Given for example

staticFiles("/", File(pathToStaticFiles), index = "index.html") {
    default("index.html")
}

or

singlePageApplication {
    filesPath = pathToStaticFiles
}

and a request for a directory, for example GET / , the plugin attempts to respond twice, first with the index, then the default. It seems to behave normally apart from the double 200 OK: GET - / in the logs, but in my case it constantly causes UnsupportedOperationException: Headers can no longer be set because response was already completed for call when another plugin tries to append headers during the second respond call.

My guess is that there's a missing if(isHandled) return around here: https://github.com/ktorio/ktor/blob/46ac5de2495c399453e11e3078404bedcdad72ba/ktor-server/ktor-server-core/jvm/src/io/ktor/server/http/content/StaticContent.kt#L464

Analogous to https://youtrack.jetbrains.com/issue/KTOR-872, it would be very useful to have linuxArm64 support on ktor server.

While reading testApplication, I don't understand why block() is executed before testApplication.start()?

    @KtorDsl
    public fun testApplication(
        parentCoroutineContext: CoroutineContext = EmptyCoroutineContext,
        block: suspend ApplicationTestBuilder.() -> Unit
    ) {
        val builder = ApplicationTestBuilder()
            .apply {
                runBlocking(parentCoroutineContext) {
                    if (parentCoroutineContext != EmptyCoroutineContext) {
                        environment {
                            this.parentCoroutineContext = parentCoroutineContext
                        }
                    }
                    block()
                }
            }
    
        val testApplication = TestApplication(builder)
        testApplication.start()
        testApplication.stop()
    }

Now it's impossible to provide dependency

Protobuf does not support empty lists or nullable values. As such if the body of the response from the server is a object like:

data class MyClass(val aList: List<String> = emptyList())

and the list is empty, the body of the response will be empty.

This implies that when deserializing:

// io.ktor:ktor-serialization:2.3.2

package io.ktor.serialization
// ...
public suspend fun List<ContentConverter>.deserialize(
    body: ByteReadChannel,
    typeInfo: TypeInfo,
    charset: Charset
): Any {
   
    val result = asFlow()
        .map { converter -> converter.deserialize(charset = charset, typeInfo = typeInfo, content = body) }
        .firstOrNull { it != null || body.isClosedForRead }

    return when {
        result != null -> result
        !body.isClosedForRead -> body
        typeInfo.kotlinType?.isMarkedNullable == true -> NullBody
        else -> throw ContentConvertException("No suitable converter found for $typeInfo")
    }
}

The channel is always closed for read, making result a null and always going for the the else branch of the when .

I am not sure if it happens only to the test engine provided by the test server, but given the code running above, I assume not.

We have enabled WebSocket compression by default:

        install(WebSockets) {
            extensions {
                install(WebSocketDeflateExtension)
            }
        }

And now we're getting the following exception:

java.util.zip.DataFormatException invalid distance too far back
	at java.base/java.util.zip.Inflater.$$YJP$$inflateBytesBytes(Native Method)
	at java.base/java.util.zip.Inflater.inflateBytesBytes(Inflater.java)
	at java.base/java.util.zip.Inflater.inflate(Inflater.java:378)
	at io.ktor.websocket.internals.DeflaterUtilsKt.inflateFully(DeflaterUtils.kt:50)
	at io.ktor.websocket.WebSocketDeflateExtension.processIncomingFrame(WebSocketDeflateExtension.kt:140)
	at io.ktor.websocket.DefaultWebSocketSessionImpl.processIncomingExtensions(DefaultWebSocketSession.kt:325)
	at io.ktor.websocket.DefaultWebSocketSessionImpl.access$processIncomingExtensions(DefaultWebSocketSession.kt:70)
	at io.ktor.websocket.DefaultWebSocketSessionImpl$runIncomingProcessor$1.invokeSuspend(DefaultWebSocketSession.kt:204)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:233)
	at kotlinx.coroutines.DispatchedTaskKt.resumeUnconfined(DispatchedTask.kt:189)
	at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:161)
	at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:474)
	at kotlinx.coroutines.CancellableContinuationImpl.completeResume(CancellableContinuationImpl.kt:590)
	at kotlinx.coroutines.channels.BufferedChannelKt.tryResume0(BufferedChannel.kt:2896)
	at kotlinx.coroutines.channels.BufferedChannelKt.access$tryResume0(BufferedChannel.kt:1)
	at kotlinx.coroutines.channels.BufferedChannel$BufferedChannelIterator.tryResumeHasNext(BufferedChannel.kt:1689)
	at kotlinx.coroutines.channels.BufferedChannel.tryResumeReceiver(BufferedChannel.kt:642)
	at kotlinx.coroutines.channels.BufferedChannel.updateCellSend(BufferedChannel.kt:458)
	at kotlinx.coroutines.channels.BufferedChannel.access$updateCellSend(BufferedChannel.kt:36)
	at kotlinx.coroutines.channels.BufferedChannel.send$suspendImpl(BufferedChannel.kt:3089)
	at kotlinx.coroutines.channels.BufferedChannel.send(BufferedChannel.kt)
	at io.ktor.websocket.RawWebSocketJvm$1.invokeSuspend(RawWebSocketJvm.kt:68)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
	at java.base/java.lang.Thread.run(Thread.java:833)

kotlinx-html has dropped the onTagError method (https://github.com/Kotlin/kotlinx.html/pull/224). The ktor integration currently relies on this. When updating the html dependency, errors cannot be handled properly anymore:

java.lang.NoSuchMethodError: 'void kotlinx.html.TagConsumer.onTagError(kotlinx.html.Tag, java.lang.Throwable)'
	at io.ktor.server.html.RespondHtmlKt.respondHtml(RespondHtml.kt:68)
	at io.ktor.server.html.RespondHtmlTemplateKt.respondHtmlTemplate(RespondHtmlTemplate.kt:21)
	at io.ktor.server.html.RespondHtmlTemplateKt.respondHtmlTemplate$default(RespondHtmlTemplate.kt:15)
	at

Ktor sets main class using deprecated method because of dependency on shadow plugin v6, that results in following warning:

The JavaApplication.setMainClassName(String) method has been deprecated. This is scheduled to be removed in Gradle 8.0. Use #getMainClass().set(...) instead. See https://docs.gradle.org/7.6/dsl/org.gradle.api.plugins.JavaApplication.html#org.gradle.api.plugins.JavaApplication:mainClass for more details.

Shadow plugin v7 already released, so it can be updated in Ktor plugin if there are no minimal gradle version requirements.

I am in the process of migrating from retrofit to ktor. I am using OkHttp to keep the interceptor logic. One of these interceptors requires that the "contentType" field is not null for logging.

I use the FormDataContent class to pass parameters to the form according to https://ktor.io/docs/request.html#form_parameters:

// class io.ktor.client.request.forms.FormDataContent

public class FormDataContent(
    public val formData: Parameters
) : OutgoingContent.ByteArrayContent() {
    private val content = formData.formUrlEncode().toByteArray()

    override val contentLength: Long = content.size.toLong()
    override val contentType: ContentType = ContentType.Application.FormUrlEncoded.withCharset(Charsets.UTF_8) <----- setting form content type

    override fun bytes(): ByteArray = content
}

But when converting to OkHtpp classes, the "contentType" parameter is lost:

// class io.ktor.client.engine.okhttp.OkHttpEngine

@OptIn(DelicateCoroutinesApi::class)
internal fun OutgoingContent.convertToOkHttpBody(callContext: CoroutineContext): RequestBody = when (this) {
    is OutgoingContent.ByteArrayContent -> bytes().let {
        it.toRequestBody(null, 0, it.size) <----- why null?
    }
    is OutgoingContent.ReadChannelContent -> StreamRequestBody(contentLength) { readFrom() }
    is OutgoingContent.WriteChannelContent -> {
        StreamRequestBody(contentLength) { GlobalScope.writer(callContext) { writeTo(channel) }.channel }
    }
    is OutgoingContent.NoContent -> ByteArray(0).toRequestBody(null, 0, 0)
    else -> throw UnsupportedContentTypeException(this)
}

please tell me is this behavior is correct? how can i fix it?

On Kotlin 1.8.10 and 1.8.20-RC2, the following error occurs during :linkReleaseExecutableMingwX64

Compilation failed: Internal compiler error: no implementation found for FUN DEFAULT_PROPERTY_ACCESSOR name:<get-parent> visibility:public modality:ABSTRACT <> ($this:kotlinx.coroutines.Job) returnType:kotlinx.coroutines.Job?
when building itable for CLASS INTERFACE name:Job modality:ABSTRACT visibility:public superTypes:[kotlin.coroutines.CoroutineContext.Element]
implementation in CLASS CLASS name:ChannelJob modality:FINAL visibility:private superTypes:[io.ktor.utils.io.ReaderJob; io.ktor.utils.io.WriterJob; kotlinx.coroutines.Job]
at Z:/buildAgent/work/8d547b974a7be21f/ktor-io/common/src/io/ktor/utils/io/Coroutines.kt (156:1)
CLASS CLASS name:ChannelJob modality:FINAL visibility:private superTypes:[io.ktor.utils.io.ReaderJob; io.ktor.utils.io.WriterJob; kotlinx.coroutines.Job]

 * Source files: 
 * Compiler version info: Konan: 1.8.20-RC2 / Kotlin: 1.8.20
 * Output kind: PROGRAM

But, linkDebugExecutableMingwX32 works.

I only tried on Linux

https://ktor.io/docs/oauth.html is using sessions without installing the plugin. It will lead to the runtime error

Create new project with configuration in YAML

In Application.kt file there is a comment about application.conf references for the main function, but configuration is in application.yaml file.

    val app: TestApplication = TestApplication {
        environment {
            val twiceNestedValue = MapApplicationConfig().apply { put("ktor.database.jdbcURL", TestDbContainer.getDbURL()) }
            val onceNestedValue = MapApplicationConfig().apply { put("database.jdbcURL", TestDbContainer.getDbURL()) }
            log.info("twiceNestedValue: ${twiceNestedValue.toMap()}")
            log.info("onceNestedValue: ${onceNestedValue.toMap()}")
        }
    }

Output:

INFO  ktor.test - twiceNestedValue: {ktor={database={}}}
INFO  ktor.test - onceNestedValue: {database={jdbcURL=jdbc:postgresql://localhost:49250/somedb?loggerLevel=OFF}}

Creating config at runtime in this manner is the JetBrains suggested solution https://stackoverflow.com/a/75590178/9397815

Let's consider this request:

METHOD: HttpMethod(value=POST)
COMMON HEADERS
-> Accept: application/json
-> Accept-Charset: UTF-8
-> Authorization: Bearer fake:0
CONTENT HEADERS
-> Content-Length: 27
-> Content-Type: application/json
BODY Content-Type: application/json
BODY START
{"name":"A new department"}
BODY END

Using Ktor 2.2.1, using the Authentication plugin with bearer {}, receiving this request instantly returns a 400 Bad Request with no body. Because I used the ContentNegotiation plugin as well, the error message I got was

Unknown: 400 Bad Request with no provided body

with a previously logged line

TRACE i.k.s.p.c.ContentNegotiation - Skipping because the type is ignored.

leading me in a completely wrong direction.

I'm not sure having a colon in a Bearer token is even allowed by the RFCs, but if it isn't the error message should be clearer that this is the issue. For example, returning an error message, or logging the clear explanation.
So far the only actually relevant log I could find was:

TRACE i.k.s.auth.Authentication - Trying to authenticate /test with null

which doesn't help.

When BearerAuthProvider calls AuthTokenHolder.setToken, it passes a block that is configurable by ktor consumers (unliked BasicAuthProvider and DigestAuthProvider). When this block throws an exception, it results in the skipping of newDeferred.complete(newTokens) and refreshTokensDeferred.value = null, which causes the next call to setToken to hang indefinitely on deferred.await(). Here are the relevant blocks of code:

https://github.com/ktorio/ktor/blob/5f5947ebae390cece2c1a22ac92d5651d764471d/ktor-client/ktor-client-plugins/ktor-client-auth/common/src/io/ktor/client/plugins/auth/providers/BearerAuthProvider.kt#L138-L143

https://github.com/ktorio/ktor/blob/5f5947ebae390cece2c1a22ac92d5651d764471d/ktor-client/ktor-client-plugins/ktor-client-auth/common/src/io/ktor/client/plugins/auth/providers/AuthTokenHolder.kt#L59-L69

A workaround for this is for ktor consumers to wrap their refreshTokens logic in a try block that catches Throwable like this:

install(Auth) {
  bearer {
    loadTokens {
      // ...
    }
    refreshTokens {
      try {
        // ...
      } catch (t: Throwable) {
        // don't allow this to bubble up
      }
    }
  }
}

Summary

Ktor 2.3.0 client under Javascript is unable to stream responses from a server. The same code works correctly with Ktor running under the JVM.

Steps to reproduce

1. Configure a server to emit responses as a stream
2. Configure a ktor client in Javascript to read the response as a stream (e.g. response.bodyAsChannel() and reading response line by line)

With the attached sample application

1. Run the server ./gradlew :server-app:run and make sure you have a console to look at the server log output
2. Launch your browser and open the developer tools
3. Visit http://localhost:8080
4. Compare the logs from the client and server

Results

Expected

Javascript client reads stream as each item is written

Actual

Javascript buffers until entire response is complete before actually reading it.

Example output from the test application
Server log output:

Writing H at 2023-04-29T16:56:47.240733Z
Writing e at 2023-04-29T16:56:47.750986Z
Writing l at 2023-04-29T16:56:48.256360Z
Writing l at 2023-04-29T16:56:48.761753Z
Writing o at 2023-04-29T16:56:49.264868Z
Writing   at 2023-04-29T16:56:49.765610Z
Writing w at 2023-04-29T16:56:50.270883Z
Writing o at 2023-04-29T16:56:50.776197Z
Writing r at 2023-04-29T16:56:51.283105Z
Writing l at 2023-04-29T16:56:51.788978Z
Writing d at 2023-04-29T16:56:52.292238Z

Browser log output:

Received H at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received e at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received l at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received l at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received o at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received w at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received o at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received r at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received l at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received d at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)

Notes

I reproduced this with Ktor 2.3.0. I have not tested older versions to see whether this is a long standing bug or a regression.

We have an API that generates a document served by AWS cloudfront and returns a URL to download the document when it is complete. The problem is that the URL has an extra leading & in the query as in: "?&Expires". When the Ktor client makes the call it drops the leading ampersand which in my mind is correct since it really has no meaning. But unfortunately without the ampersand the call fails.

There is no way to force Ktor not to drop the ampersand.

Cloudfront should not be generating a URL like this and even worse should not be treating the presence as significant and I have a back end person looking into it, but as it stands now we have to have the extra ampersand and will have to hack and expect/actual API to use the platform specific Http Client.

After testing the KTOR-5650 fix (https://github.com/ktorio/ktor/pull/3456) in Electron the check turned out to return false even though we have the Node API. In Electron at this point there can be two process objects and the correct one is defined on window.process, not just process. So we'd have to first detect which process object to use and after that the remaining .versions.node check can be done.

Expected behavior: Coroutine cancellation leads WebSocket to be closed by Normal Closure (1000) reason or like so

Actual behavior: Any coroutine completion with non-null cause leads WebSocket to be closed by Internal Error (1011) reason

Also, this reason for closing is not supported by browsers, which causes an invalid access error, leaving the WebSocket open.

I am attempting to migrate an existing netty based application to ktor.
One of its features is handling large file uploads (100mb -> 10Gb).
We are handle sensitive personal information and would prefer not write anything to disk.
I can't see a way to implement this using the existing multipart file upload support in Ktor.

Is there a way to directly access a ReadByteChannel for the file and avoid the tmp disk write?

For example, in HOCON I can write the following:

value {
  my = "My value"
}

config {
  database {
    value = ${value.my}
  }
}

If I write similar config file in YAML:

value:
  my: "My value"

config:
  database:
    value: $value.my

It will be failing to read $value.my assuming it's an environment variable

Steps to reproduce:

1. Create a Ktor application

import io.ktor.server.application.*
import io.ktor.server.cio.*
import io.ktor.server.routing.*

fun main(args: Array<String>) {
    EngineMain.main(args)
}

@Suppress("unused")
fun Application.main() {
    routing {
    }
}

2. Create two external config files

application.conf

ktor {
  development = false

  deployment {
    host = "0.0.0.0"
    port = 8080
    shutdownGracePeriod = 2000
    shutdownTimeout = 60000
  }

  application {
    modules = [package.MainKt.main]
  }
}

extra.conf

// Ktor can't load additional configuration without "ktor" definition
//ktor {
//  deployment {
//  }
//}

app {
  test = "conf"
}

3. Run the app using external configuration files
   -config=application.conf -config=extra.conf

The result

Actual
It fails with Exception in thread "main" com.typesafe.config.ConfigException$Missing: extra.conf: 1: No configuration setting found for key 'ktor'

Expected
The app should run since the ktor configuration is defined in the application.conf file

As a workaround, it works when empty ktor and deployment configurations are added to the extra.conf file.

KotlinxSerializationConverter#deserialize() “hides” actual deserialization error messages:

// KotlinxSerializationConverter.kt
fun deserialize(): Any? {
  try {
    // Deserialize and return if successful.
  } catch (cause: Throwable) {
    throw JsonConvertException("Illegal input", cause)
  }
}

While the message will be available in the cause of the thrown JsonConvertException , it causes the Logging plugin to display some pretty useless logs:

// Logging.kt
fun logResponseException(log: StringBuilder, request: HttpRequest, cause: Throwable) {
  log.append("RESPONSE ${request.url} failed with exception: $cause")
}

The result is logs like this, which unfortunately doesn't provide much value:

RESPONSE https://host/path failed with exception: io.ktor.serialization.JsonConvertException: Illegal input

Is there a way we can provide cause.message alongside the "Illegal input" message from KotlinxSerializationConverter? Perhaps just something like this:

throw JsonConvertException("Illegal input: ${cause.message}", cause)

Currently, upon receipt of an `X-Forwarded-For` header, the XForwardedHeaders plugin sets the remoteHost origin property:

https://github.com/ktorio/ktor/blob/e26794418af9bd2e0f38dc25cd938868ffa71fc5/ktor-server/ktor-server-plugins/ktor-server-forwarded-header/jvmAndNix/src/io/ktor/server/plugins/forwardedheaders/XForwardedHeaders.kt#L172

If the value extracted from the header is an address rather than a host, Ktor should be setting the remoteAddress property.

The KDocs for these two properties are:

/** Client address or host name if it can be resolved. ... */
public val remoteHost: String

and

/** Client address. ... */
public val remoteAddress: String

which clearly implies that for an address, these two values should be the same. The kdocs for remoteAddress even specify that it can be overwridden via the forwarded headers.

Therefore, client code that wants the remote address should clearly be using remoteAddress, and yet when the XForwardedHeaders plugin is used and the headers resolve to an address, the remoteHost will contain the resolved remote address, but remoteAddress will not.

Hi, I'm new to Ktor.
please understand if my question is wrong.

when I use session like below

install(Sessions) {
        cookie<User>(
            USER_COOKIE_NAME,
            directorySessionStorage(File(".sessions"), cached = true)
        )

I set session on signin api is called.
and client saved the sessionId and delivered the sessionId to server on getUser api like below
and server response with Set-Cookie header even getUser api controller didn't set session.

REQUEST: http://192.168.224.95:8080/getUser
METHOD: HttpMethod(value=POST)
COMMON HEADERS
-> KEY: Header test
-> Accept: application/json
-> Accept-Charset: UTF-8
-> Cookie: user=4879fe5aee4c5b3d;
CONTENT HEADERS
BODY Content-Type: application/json
BODY Content-Type: application/json; charset=UTF-8
BODY START
RESPONSE: 200 OK
METHOD: HttpMethod(value=POST)
FROM: http://192.168.224.95:8080/getUser
COMMON HEADERS
-> Connection: keep-alive
-> Content-Length: 21
-> Content-Type: application/json; charset=UTF-8
-> Set-Cookie: simple-user=4879fe5aee4c5b3d; Max-Age=604800; Expires=Thu, 20 Aug 2020 15:57:08 GMT; Path=/; HttpOnly; $x-enc=URI_ENCODING
-> X-Android-Received-Millis: 1597334141402
-> X-Android-Response-Source: NETWORK 200
-> X-Android-Selected-Protocol: http/1.1
-> X-Android-Sent-Millis: 1597334137063
{"id":"a","name":"b"}
BODY END

so, I checked which part add Set-Session
and found this code on io.ktor.sessions.Sessions

pipeline.intercept(ApplicationCallPipeline.Features) {
                val providerData = sessions.providers.associateBy({ it.name }) {
                    it.receiveSessionData(call)
                }
                val sessionData = SessionData(sessions, providerData)
                call.attributes.put(SessionKey, sessionData)
            }

in my past experience, Set-Session is added only when session is set.
But, my experience is not long, so, kindly understand if I'm wrong.
I would like to ask if this is intended behavior.

Thanks to read.
Please let me know if further information is required.

I have Ktor server in an Android app and I use it to receive files. I would like to display a progress bar in my app for the duration it will take for the receiving file to arrive.

Some sort of a listener with bytesSent/bytesTotal would be perfect

Just to clarify, I am not interested in listening to upload progress from a client, but the time it takes to receive a files from the server.

Right now my incoming files handling code looks like this:

// 1
call.receiveMultipart().forEachPart { part ->
    // 2
    when (part) {
        is PartData.FileItem -> {
            val input = part.provider()
            // Read from input by chunks
        }
        else -> {}
    }
}

I would like to listen to the progress between (1) and (2). Right now (1) is called, then the file is fully uploaded to the server, and then (2) is called.

Platform: JVM

We recently observed our production application servers locking up and becoming unresponsive due to legacy clients making a large number of web socket upgrade requests to a URI for which a previous route no longer existed. This is equivalent to a denial-of-service attack. We traced the issue back to the simple reproduction below, which also exposes a Ktor web socket client ambiguity on how to handle 404 and other connection issues: the documentation is not clear about what happens in those cases. If catching an exception is the way to go, as in the reproduction below, please update the documentation.

The issue appears to be that the server pipeline for web socket upgrade requests never completes if the client's upgrade request is sent to a URI for which no web socket handler route exists. The attached reproduction starts a server and then attempts to make 1,000 web socket connections. As is, this succeeds and at the end of the test run, roughly 10 coroutines exist on the server.

If the configured route is changed from /test/{id} to /test, the server locks up after processing ~200 requests, meaning it doesn't become unresponsive immediately, but only past a certain load. Over 4,000 suspended coroutines exist on the server at the end of the failed test run.

The workaround we're testing right now is to accept and immediately drop the connections to the known legacy route, but that doesn't resolve the issue that a potential attacker who knows that Ktor with web socket support is running on our servers could craft a large number of web socket requests to a different non-existing route and cause application servers to become unresponsive!

In HOCON there are 3 scenarios of variable declaration:

a) With default value

myValue = "default value"
myValue = ${?MY_ENV_VAR}

myValue equals "default value" if one doesn't provide MY_ENV_VAR

b) Without default value and with question mark

myValue = ${?MY_ENV_VAR}

myValue equals the value of MY_ENV_VAR or does not exist otherwise (.getProperty returns null)

c) Without default value and with no question mark

myValue = $MY_ENV_VAR

myValue equals the value of MY_ENV_VAR which must be present (.getProperty throws an exception in attempt to read MY_ENV_VAR)

There are only 2 equivalents in YAML:
a) With default value

myValue = "$MY_ENV_VAR:default value"

myValue equals "default value" if one doesn't provide MY_ENV_VAR

c) With no default value

myValue = $MY_ENV_VAR

myValue equals the value of MY_ENV_VAR which must be present (.getProperty throws an exception in attempt to read MY_ENV_VAR)

The only way to achieve "b" in YAML is to write ugly code like the following introducing an extra "NULL" string (Or any other token that one believes will never present in real values):

my:
  config: "$MY_ENV_VAR:NULL"

And then handling it like the following.

fun ApplicationConfig.yamlStringPropertyOrNull(path: String): String? = propertyOrNull(path)
    ?.getString()
    ?.takeIf { it != "NULL" }

fun ApplicationConfig.yamlListProperty(path: String): List<String> = property(path)
    .getList()
    .filter { it != "NULL" }

val myValue get(): MyConfig = conf.yamlStringPropertyOrNull("my.config")

When using the example here I get a java.lang.NullPointerException when running the test. Am I doing something wrong? My hope is to be able to test my WebSocket client of my KMP-app and I got the impression that this might be able to help me?

Using Ktor 2.2.4 and Kotlin 1.8.10.

Here's the stacktrace of the crash. Thanks in advance.

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

java.lang.NullPointerException
	at io.ktor.server.testing.TestApplicationEngineJvmKt$handleWebSocketConversationNonBlocking$5.invokeSuspend(TestApplicationEngineJvm.kt:73)
	(Coroutine boundary)
	at io.ktor.server.testing.TestApplicationEngineJvmKt.handleWebSocketConversationNonBlocking(TestApplicationEngineJvm.kt:68)
	at io.ktor.server.testing.client.TestHttpClientEngineBridge.runWebSocketRequest(TestHttpClientEngineBridgeJvm.kt:34)
	at io.ktor.server.testing.client.TestHttpClientEngine.execute(TestHttpClientEngine.kt:50)
	at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:99)
	at io.ktor.client.engine.HttpClientEngine$DefaultImpls.executeWithinCallContext(HttpClientEngine.kt:100)
	at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:70)
	at io.ktor.client.plugins.HttpSend$DefaultSender.execute(HttpSend.kt:138)
	at io.ktor.client.plugins.HttpRedirect$Plugin$install$1.invokeSuspend(HttpRedirect.kt:64)
	at io.ktor.client.plugins.HttpCallValidator$Companion$install$3.invokeSuspend(HttpCallValidator.kt:151)
	at io.ktor.client.plugins.HttpSend$Plugin$install$1.invokeSuspend(HttpSend.kt:104)
	at io.ktor.client.plugins.websocket.WebSockets$Plugin$install$1.invokeSuspend(WebSockets.kt:169)
	at io.ktor.client.plugins.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:130)
	at io.ktor.client.plugins.HttpRequestLifecycle$Plugin$install$1.invokeSuspend(HttpRequestLifecycle.kt:38)
	at io.ktor.client.HttpClient.execute$ktor_client_core(HttpClient.kt:191)
	at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:108)
	at io.ktor.client.plugins.websocket.BuildersKt.webSocket(builders.kt:241)
	at ModuleTest$testConversation$1.invokeSuspend(ModuleTest.kt:15)
	at io.ktor.server.testing.TestApplicationKt$testApplication$builder$1$1.invokeSuspend(TestApplication.kt:288)
Caused by: java.lang.NullPointerException
	at io.ktor.server.testing.TestApplicationEngineJvmKt$handleWebSocketConversationNonBlocking$5.invokeSuspend(TestApplicationEngineJvm.kt:73)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)

The Rate limit provider with name RateLimitName (name=KTOR_NO_NAME_RATE_LIMITER) is not configured or a similar error doesn't explain that the RateLimit plugin must be installed if it wasn't and that its installation must go before the one of the routing.

In case the plugin is set up to have separate limiter per user, this can cause unnecessary memory usage.

https://github.com/ktorio/ktor/blob/1d110bc08f0bdebb96d5370dd1a94ea65e64ec2e/ktor-shared/ktor-websockets/common/src/io/ktor/websocket/DefaultWebSocketSession.kt#L78

        val calendar = Calendar.getInstance(TimeZone.getTimeZone("Europe/Berlin"))
        val timestamp = calendar.toDate(null).timestamp // Equals to GMTDate().timestamp

After creating a new GMTDate instance from this timestamp, a different date is created.

File Websockets.kt contains own LOGGER

internal val LOGGER = KtorSimpleLogger("io.ktor.client.plugins.websocket.WebSockets")

Which can't be configured.

Implementation of KtorSimpleLoggerJs.kt

@Suppress("FunctionName")
public actual fun KtorSimpleLogger(name: String): Logger = object : Logger {
    override fun error(message: String) {
        console.error(message)
    }

    override fun error(message: String, cause: Throwable) {
        console.error("$message, cause: $cause")
    }

    override fun warn(message: String) {
        console.warn(message)
    }

    override fun warn(message: String, cause: Throwable) {
        console.warn("$message, cause: $cause")
    }

    override fun info(message: String) {
        console.info(message)
    }

    override fun info(message: String, cause: Throwable) {
        console.info("$message, cause: $cause")
    }

    override fun debug(message: String) {
        console.info("DEBUG: $message")
    }

    override fun debug(message: String, cause: Throwable) {
        console.info("DEBUG: $message, cause: $cause")
    }

    override fun trace(message: String) {
        console.info("TRACE: $message")
    }

    override fun trace(message: String, cause: Throwable) {
        console.info("TRACE: $message, cause: $cause")
    }
}

It floods logs with unnecessary and useless messages (why not console.trace?):

TRACE: Sending WebSocket request [object Object]

This logger should be configurable with Logging plugin.

In the engine there are several places which check first if (PlatformUtils.IS_BROWSER) but it would be better if those checked for if (PlatformUtils.IS_NODE) first (i.e. flip the if and else blocks) because for Electron IS_NODE and IS_BROWSER can both be true at the same time, but the browser fetch API is very crippled in comparison to the node-fetch based solution. For example you can't read response headers without explicitly setting up CORS headers in the backend.

In general, instead of having two Booleans (which can both be true) it would be better to have an enum and a single place in the code where the decision of prioritizing Node vs. browser behavior happens. Otherwise such "bugs" of prioritizing crippled browser APIs will happen all over the place and harm Electron integration.

When the server sends a PING frame, the Ktor client replies with 2 PONG frames.

Maybe the Java web socket client implementation already deals with PING/PONG automatically, and Ktor adds another layer of PING handling, resulting in 2 PONGs, but I haven't checked the code.

It is technically valid to send unsolicited PONG frames as per the web socket specification, but this is not quite what I would expect, and it's breaking Autobahn TestSuite, marking Ktor as failing ping/pong tests.

I'm experiencing that the DarwinClientEngine deadlocks on any http requests since 2.2.2 if they are started from the main thread. If I downgrade to 2.2.1 everything works fine.

deadlocks:

HttpClient(
    Darwin.create {
        pipelining = true
    }
)

println("making request..")
val result = httpClient().get("https://www.google.com")
println("status: $result.status")

works:

HttpClient(
    Darwin.create {
        pipelining = true
    }
)

launch(Dispatchers.Default) {
    println("making request..")
    val result = httpClient().get("https://www.google.com")
    println("status: $result.status")
}

We are using BearerAuthProvider and experiencing unexpected consecutive refresh token calls in this particular scenario.

Here the sequence of the calls with url, response code, response time and token. The calls are in response order.

1. /url1 -> 401 - response time: 310 ms - uses token1
2. /url2 -> 401 - response time: 276 ms - uses token1
3. /url3 -> 401 - response time: 322 ms - uses token1
4. /refresh -> 200 - response time: 390 ms - refresh token 1 to token 2
5. /url1 -> 200 - response time: 168 ms - uses token 2
6. /url2 -> 200 - response time: 179 ms - uses token 2
7. /url3 -> 200 - response time: 189 ms - uses token 2
8. /url4 -> 401 - response time: 264 ms - uses token 1
9. /refresh -> 200 - response time: 168 ms - refresh token 2 to token 3
   ...

What we see is that:

• url1, url2, url3 are queued for retry and retried after step 4 in which the refresh token is done. These retries succeed.
• url4 is started when the refresh token is in process but the 401 is received after the refresh token finishes. The 401 failure of url4 leads to another refresh token. We were expecting that this call was going to be retried immediately without a refresh token, just using token2 (refreshed token received at step 4).

The situation gets even worse if we amplify this behaviour to a higher number of calls: e.g. this could lead to url4 failing twice with 401 and this call will be lost and never complete.

realm should be set earlier in the function

https://github.com/ktorio/ktor/blob/ebaf9f16786566a3061ebd4fc9de5523c0291025/ktor-client/ktor-client-plugins/ktor-client-auth/common/src/io/ktor/client/plugins/auth/providers/DigestAuthProvider.kt#L165

should be moved to before line 154. This fixes the issues for me.

Currently in JavaHttpWebSocket.onError, Ktor throws a WebSocketException with just the message from the cause:

    override fun onError(webSocket: WebSocket, error: Throwable) {
        val cause = WebSocketException("${error.message}")
        _incoming.close(cause)
        _outgoing.cancel()
        socketJob.complete()
    }

https://github.com/ktorio/ktor/blob/main/ktor-client/ktor-client-java/jvm/src/io/ktor/client/engine/java/JavaHttpWebSocket.kt#L186

This is not a nice solution in general, because exception messages don't completely inform about the problem. It's the combination of exception type + message that gives the full insight. For instance, with FileNotFoundException: /tmp/foo, we don't want to just bubble up the message /tmp/foo, it doesn't really explain the problem.

As a more realistic example, I'm getting a "null" message from Ktor because some exception in the Java web socket has no message, which makes this stacktrace pretty pointless:

Caused by: org.hildan.krossbow.websocket.WebSocketException: error in Ktor's websocket: null	
at org.hildan.krossbow.websocket.ktor.KtorWebSocketConnectionAdapter$incomingFrames$3.invokeSuspend(KtorWebSocketClient.kt:60)	
•••
at org.hildan.krossbow.websocket.test.TestUtilsJvmKt.runSuspendingTest(TestUtilsJvm.kt:8)	
at org.hildan.krossbow.websocket.test.TestUtilsJvmKt.runSuspendingTest$default(TestUtilsJvm.kt:8)	
at org.hildan.krossbow.websocket.test.autobahn.AutobahnClientTestSuite.runAutobahnTestCase(AutobahnClientTestSuite.kt:200)	
at org.hildan.krossbow.websocket.test.autobahn.AutobahnClientTestSuite.autobahn_1_1_7_echo_text_payload(AutobahnClientTestSuite.kt:54)	
•••
Caused by: io.ktor.client.plugins.websocket.WebSocketException: null	
at (Coroutine boundary.()	
•••
Caused by: io.ktor.client.plugins.websocket.WebSocketException: null	
at io.ktor.client.engine.java.JavaHttpWebSocket.onError(JavaHttpWebSocket.kt:186)	

And the chain of causes stops there. See this build scan for a full example stack trace.

Could you please add the cause here to the WebSocketException?

I've stumbled upon improper code generated from swagger spec, roughly equivalent to:

append("key[]", listOf("value0", "value1", "etc"))

Although it is easy to convert to something like:

listOf("value0", "value1", "etc").forEach { append("key[]", it) }

Seems presence of catch-all append<T> is confusing users:

    /**
     * Appends a pair [key]:[value] with optional [headers].
     */
    @InternalAPI
    public fun <T : Any> append(key: String, value: T, headers: Headers = Headers.Empty) {
        parts += FormPart(key, value, headers)
    }

I propose adding additional append overload to support List<String> and optionally List<Number> parameters:

    /**
     * Appends a pair [key]:[values] with optional [headers].
     */
    public fun append(key: String, values: List<String>, headers: Headers = Headers.Empty) {
        require(key.endsWith("[]")) {
            "Array parameter must be suffixed with square brackets ie `key[]`"
        }
        values.forEach { value ->
            parts += FormPart(key, value, headers)
        }
    }
    

We are building a Kotlin Native project for desktop. We decided to use ktor-client-curl, as it supports both Mac and Windows. On the other hand, currently the client fails on Windows with the following exception when using any HTTPS URLs:

io.ktor.client.engine.curl.CurlIllegalStateException: TLS verification failed for request: CurlRequestData(url='https://google.com', method='GET', content: 0 bytes). Reason: SSL peer certificate or SSH remote key was not OK
io.ktor.client.engine.curl.CurlIllegalStateException: TLS verification failed for request: CurlRequestData(url='https://google.com', method='GET', content: 0 bytes). Reason: SSL peer certificate or SSH remote key was not OK
	at kotlin.Throwable#<init>(Unknown Source)
	at kotlin.Exception#<init>(Unknown Source)
	at kotlin.RuntimeException#<init>(Unknown Source)
	at kotlin.IllegalStateException#<init>(Unknown Source)
	at io.ktor.client.engine.curl.CurlIllegalStateException#<init>(Unknown Source)
	at io.ktor.client.engine.curl.internal.CurlMultiApiHandler.collectFailedResponse#internal(Unknown Source)
	at io.ktor.client.engine.curl.internal.CurlMultiApiHandler.processCompletedEasyHandle#internal(Unknown Source)
	at io.ktor.client.engine.curl.internal.CurlMultiApiHandler.handleCompleted#internal(Unknown Source)
	at io.ktor.client.engine.curl.internal.CurlMultiApiHandler#perform(Unknown Source)
	at io.ktor.client.engine.curl.CurlProcessor.$runEventLoop$lambda$3COROUTINE$17.invokeSuspend#internal(Unknown Source)
	at kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(Unknown Source)
	at kotlinx.coroutines.DispatchedTask#run(Unknown Source)
	at io.ktor.util.MultiWorkerDispatcher.$workerRunLoop$lambda$4COROUTINE$80.invokeSuspend#internal(Unknown Source)
	at kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(Unknown Source)
	at kotlinx.coroutines.DispatchedTask#run(Unknown Source)
	at kotlinx.coroutines.EventLoopImplBase#processNextEvent(Unknown Source)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal(Unknown Source)
	at kotlinx.coroutines#runBlocking(Unknown Source)

It looks like it's not using the certificate store of Windows, and for me it also failed to detect the ca-cert.pem file regardless where I copied it (see https://github.com/ktorio/ktor/issues/671#issuecomment-619832512 for suggestions).

It would be great if the CURLOPT_CAINFO and CURLOPT_CAPATH options would be configurable for the Ktor Curl client as this way I could directly provide the path to the ca-cert.pem file.

Description
After updating from KTOR 2.1.2 to 2.2.2 and adding HttpCache I started to receive bug reports from my testers regarding a GZIP magic invalid: 12345 error. After some testing I realized that only release builds but not debug builds were affected and traced the issue to the LogLevel used, which is LogLevel.INFO for release and LogLevel.ALL for debug. Also LogLevel.HEADERS is affected by this issue.

Updating to KTOR 2.2.3 did not bring any improvements.

Setup

• KTOR 2.2.2 or 2.2.3
• OkHttp Engine on Android is broken
• Darwin Engine on iOS works as expected

The following configuration triggers the issue. It happens with and without setting the public/private storage set to my storage implementation. To ensure my storage doesn't cause the issue, I commented it out with the same result. Setting one or the other or both makes no difference.

To fix the problem, one of three things needs to happen:

• Remove the install(HttpCache) , commenting out the storages as shown does not help
• Remove the install(ContentEncoding), maybe removing GZIP is sufficient, but the server doesn't provide Deflate so I can't test this
• Set level to LogLevel.ALL

install(HttpCache) {
//  publicStorage(storage) 
//  privateStorage(storage)
}

install(ContentEncoding) {
    deflate(1.0f)
    gzip(0.9f)
}

install(Logging) {
    level = LogLevel.INFO
    logger = object  : Logger {
        override fun log(message: String) {
            println(message)
        }
    }
}

Stack trace

java.lang.IllegalStateException: GZIP magic invalid: 958
	at java.lang.reflect.Constructor.newInstance0(Native Method)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:343)
	at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$1.invoke(ExceptionUtilsJvm.kt:3)
	at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$1.invoke(ExceptionUtilsJvm.kt:1)
	at io.ktor.utils.io.ExceptionUtilsJvmKt.tryCopyException(ExceptionUtilsJvm.kt:77)
	at io.ktor.utils.io.ByteBufferChannelKt.rethrowClosed(ByteBufferChannel.kt:1)
	at io.ktor.utils.io.ByteBufferChannelKt.access$rethrowClosed(ByteBufferChannel.kt:1)
	at io.ktor.utils.io.ByteBufferChannel.setupStateForRead(ByteBufferChannel.kt:39)
	at io.ktor.utils.io.ByteBufferChannel.readAsMuchAsPossible(ByteBufferChannel.kt:34)
	at io.ktor.utils.io.ByteBufferChannel.readAvailable$suspendImpl(ByteBufferChannel.kt:1)
	at io.ktor.utils.io.ByteBufferChannel.readAvailable(ByteBufferChannel.kt:3)
	at io.ktor.utils.io.jvm.javaio.InputAdapter$loop$1.loop(Blocking.kt:126)
	at io.ktor.utils.io.jvm.javaio.InputAdapter$loop$1$loop$1.invokeSuspend(Blocking.kt:12)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:12)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:119)
	at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:11)
	at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:28)
	at io.ktor.utils.io.jvm.javaio.BlockingAdapter.submitAndAwait(Blocking.kt:16)
	at io.ktor.utils.io.jvm.javaio.BlockingAdapter.submitAndAwait(Blocking.kt:3)
	at io.ktor.utils.io.jvm.javaio.InputAdapter.read(Blocking.kt:6)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:248)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:267)
	// my app stack
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:12)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:119)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:13)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:3)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:1)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:15)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:29)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:1)
Caused by: java.lang.IllegalStateException: GZIP magic invalid: 958
	at io.ktor.util.EncodersJvmKt$inflate$1.invokeSuspend(EncodersJvm.kt:666)
	at io.ktor.util.EncodersJvmKt$inflate$1.invoke(EncodersJvm.kt:1)
	at io.ktor.util.EncodersJvmKt$inflate$1.invoke(EncodersJvm.kt:2)
	at io.ktor.utis.io.CoroutinesKt$launchChannel$job$1.invokeSuspend(Coroutines.kt:68)
	... 8 more

While playing around with logger settings I also got this error. It seems like the file I download determines which of the two exceptions I get. Downloading the same file multiple times consistently reproduces the same exception, across app restarts.

java.lang.IllegalArgumentException: max shouldn't be negative: -22207
    at java.lang.reflect.Constructor.newInstance0(Native Method)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:343)
    at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$1.invoke(ExceptionUtilsJvm.kt:103)
    at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$1.invoke(ExceptionUtilsJvm.kt:90)
    at io.ktor.utils.io.ExceptionUtilsJvmKt.tryCopyException(ExceptionUtilsJvm.kt:66)
    at io.ktor.utils.io.ByteBufferChannelKt.rethrowClosed(ByteBufferChannel.kt:2400)
    at io.ktor.utils.io.ByteBufferChannelKt.access$rethrowClosed(ByteBufferChannel.kt:1)
    at io.ktor.utils.io.ByteBufferChannel.setupStateForRead(ByteBufferChannel.kt:295)
    at io.ktor.utils.io.ByteBufferChannel.readAsMuchAsPossible(ByteBufferChannel.kt:2459)
    at io.ktor.utils.io.ByteBufferChannel.readAvailable$suspendImpl(ByteBufferChannel.kt:672)
    at io.ktor.utils.io.ByteBufferChannel.readAvailable(Unknown Source:0)
    at io.ktor.utils.io.jvm.javaio.InputAdapter$loop$1.loop(Blocking.kt:38)
    at io.ktor.utils.io.jvm.javaio.InputAdapter$loop$1$loop$1.invokeSuspend(Unknown Source:14)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
    at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:313)
    at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:201)
    at io.ktor.utils.io.jvm.javaio.BlockingAdapter.submitAndAwait(Blocking.kt:231)
    at io.ktor.utils.io.jvm.javaio.BlockingAdapter.submitAndAwait(Blocking.kt:201)
    at io.ktor.utils.io.jvm.javaio.InputAdapter.read(Blocking.kt:65)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:248)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:267)
    // my app stack
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
    at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
    at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
 Caused by: java.lang.IllegalArgumentException: max shouldn't be negative: -22207
    at io.ktor.utils.io.ByteBufferChannel.discard$suspendImpl(ByteBufferChannel.kt:1667)
    at io.ktor.utils.io.ByteBufferChannel.discard(Unknown Source:0)
    at io.ktor.util.EncodersJvmKt$inflate$1.invokeSuspend(EncodersJvm.kt:163)
    at io.ktor.util.EncodersJvmKt$inflate$1.invoke(Unknown Source:8)
    at io.ktor.util.EncodersJvmKt$inflate$1.invoke(Unknown Source:4)
    at io.ktor.utils.io.CoroutinesKt$launchChannel$job$1.invokeSuspend(Coroutines.kt:134)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) 
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) 
    at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42) 
    at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95) 
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570) 
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) 
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677) 
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664) 

Response of the server

Following file is downloaded
CE3PRO_pikachu_1gen_flowalistik.gcode

{width=70%}

Extracted this comment to a dedicated issue: https://youtrack.jetbrains.com/issue/KTOR-481/Ktor-client-logging-Add-an-option-to-filter-logged-calls#focus=Comments-27-6332247.0-0

I would like to have the ability to log debug headers like request-id trace headers while not logging sensitive Bearer token headers

I'm trying to use Ktor to talk to the REST Api of Shelly Smart Home devices, which use an lwIP webserver.

With a simple client set up like follows:

val httpClient = HttpClient(CIO)

any simple GET request fails with IllegalStateException:

Failed to parse request body: request body length should be specified,
chunked transfer encoding should be used or
keep-alive should be disabled (connection: close)

because the server neither sends Content-Length, nor Connection headers. In fact it doesn't send anything but Content-Type and Server :

HTTP/1.0 200 OK
Server: lwIP/2.1.2 (http://savannah.nongnu.org/projects/lwip)
Content-Type: application/json

I understand that this service should probably "behave better", but that's out of my control unfortunately. I debugged into io.ktor.http.cio.HttpBody.parseHttpBody and it seems to be that the issue would be solvable by defaulting to Connection: close if no Connection Header is given; and it seems this is (at least according to wikipedia) in HTTP/1.0 what the default should be; only in HTTP/1.1 connections default to being persistent.

Thanks!

Currently Boolean is not supported as a value for form data. However it could be treated the same way as Number is treated.

https://github.com/ktorio/ktor/blob/main/ktor-client/ktor-client-core/common/src/io/ktor/client/request/forms/formDsl.kt#L63

Ktor 2.1.3

package transport.ktor

import io.ktor.network.selector.*
import io.ktor.network.sockets.*
import kotlinx.coroutines.*
import kotlin.coroutines.*
import kotlin.random.*
import kotlin.test.*

class KtorReadingFromClosedSocketBugTest {
    @Test
    fun `Native - Read from a closed socket doesn't throw an exception`() {
        val address = InetSocketAddress("localhost", Random.nextInt(2_000..65_000))
        val tcp = aSocket(SelectorManager(EmptyCoroutineContext)).tcp()
        val serverSocket = tcp.bind(address)
        runBlocking {
            launch {
                serverSocket.accept()
                println("client connected")
            }
            val clientSocket = tcp.connect(address)
            launch {
                val clientReadChannel = clientSocket.openReadChannel()
                println("clientSocket.isClosed: ${clientSocket.isClosed}")
                println("clientReadChannel.isClosedForRead: ${clientReadChannel.isClosedForRead}")
                delay(2_000)
                println("clientSocket.isClosed: ${clientSocket.isClosed}")
                println("clientReadChannel.isClosedForRead: ${clientReadChannel.isClosedForRead}")
                try {
                    println("trying to read from clientReadChannel")
                    clientReadChannel.readByte()
                    println("reading from clientReadChannel succeeded^")
                } catch (e: Exception) {
                    println("reading from clientReadChannel failed")
                }
            }
            delay(1_000)
            clientSocket.close()
            println("clientSocket closed")
        }
        /*
            jvm target (expected behavior)              native target (erroneous behavior)
            ------------------------------              ----------------------------------
            client connected                            client connected
            clientSocket.isClosed: false                clientSocket.isClosed: false
            clientReadChannel.isClosedForRead: false    clientReadChannel.isClosedForRead: false
            clientSocket closed                         clientSocket closed
            clientSocket.isClosed: true                 clientSocket.isClosed: false
            clientReadChannel.isClosedForRead: true     clientReadChannel.isClosedForRead: false
            trying to read from clientReadChannel       trying to read from clientReadChannel
            reading from clientReadChannel failed       <reading from clientReadChannel never returns>
        */
    }
}

Please see Github for code.

in 1.9.0 K/JS legacy compiler is deprecated, so Ktor is broken in user projects:

e: Old Kotlin/JS compiler is no longer supported. Please migrate to the new JS IR backend

https://buildserver.labs.intellij.net/buildConfiguration/Kotlin_KotlinDev_ktor/265268196?hideProblemsFromDependencies=false&hideTestsFromDependencies=false&expandBuildProblemsSection=true&showLog=265268196_5643_683.5698&logFilter=debug&logView=flowAware

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1091

Ktor Version

ktor-http 1.1.5 and below

Feedback

Add a parameters { ... } builder DSL to replace the less discoverable Parameters.build { ... } companion object function in favor of being more consistent with the stdlib.

sequenceOf(1, 2, 3)/listOf(1, 2, 3) -> parametersOf("first", "a")
sequence { ... } -> parameters { ... }

/home/leonid/.jdks/corretto-11.0.16/bin/java -javaagent:/home/leonid/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/222.3345.118/lib/idea_rt.jar=44027:/home/leonid/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/222.3345.118/bin -Dfile.encoding=UTF-8 -classpath /home/leonid/work/js/frameworks-demo/ktor-sample/build/classes/kotlin/main:/home/leonid/work/js/frameworks-demo/ktor-sample/build/resources/main:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-content-negotiation-jvm/2.1.0/8146ef5e0b878e53003ec6c7dd8993c593e426e6/ktor-server-content-negotiation-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-auth-jwt-jvm/2.1.0/aa5373da09ad259b9483c6d727e4e3e0ffb948fd/ktor-server-auth-jwt-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-auth-jvm/2.1.0/3e54cc1f56cc79f4a52070fed610d057b7446362/ktor-server-auth-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-netty-jvm/2.1.0/dfa95e58d34d851bb80df97eef7edebcf58fa07c/ktor-server-netty-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-core-jvm/2.1.0/48db9240b2424eaadebedf06cb4b8a23badd4870/ktor-server-core-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-serialization-kotlinx-json-jvm/2.1.0/f443b9bc5f2c966026f71ab58c4afad86bf74ee5/ktor-serialization-kotlinx-json-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk8/1.7.10/d70d7d2c56371f7aa18f32e984e3e2e998fe9081/kotlin-stdlib-jdk8-1.7.10.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/ch.qos.logback/logback-classic/1.2.3/7c4f3c474fb2c041d8028740440937705ebb473a/logback-classic-1.2.3.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk7/1.7.10/1ef73fee66f45d52c67e2aca12fd945dbe0659bf/kotlin-stdlib-jdk7-1.7.10.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-jdk8/1.6.4/5bc4b0bf6fd90fc190fd2f17e919c74c6274cb71/kotlinx-coroutines-jdk8-1.6.4.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.36/6c62681a2f655b49963a5983b8b0950a6120ae14/slf4j-api-1.7.36.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-common/1.7.10/bac80c520d0a9e3f3673bc2658c6ed02ef45a76a/kotlin-stdlib-common-1.7.10.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.auth0/java-jwt/3.19.2/497c48847dcc1074d1060f9be08166aaf0853764/java-jwt-3.19.2.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.auth0/jwks-rsa/0.17.0/7c0fcd847940b6776163ef3c677d92c8e4b36798/jwks-rsa-0.17.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.googlecode.json-simple/json-simple/1.1.1/c9ad4a0850ab676c5c64461a05ca524cdfff59f1/json-simple-1.1.1.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http2/4.1.78.Final/9a4e44bebe5979b59c9bf4cd879e65c4c52bf869/netty-codec-http2-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.eclipse.jetty.alpn/alpn-api/1.1.3.v20160715/a1bf3a937f91b4c953acd13e8c9552347adc2198/alpn-api-1.1.3.v20160715.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-kqueue/4.1.78.Final/7a48e5f1cce660dd4fb5a32d9cf68d731f969555/netty-transport-native-kqueue-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-epoll/4.1.78.Final/dcb28ed02abf7bf02aa8dc9bf05f58bedcb12e9a/netty-transport-native-epoll-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-reflect/1.7.0/e660b0079fdaf744dc9cc7f6f3aa3c761ec839b0/kotlin-reflect-1.7.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.typesafe/config/1.4.2/4c40a633e7994cfb0354244efb6d03fcb11c3ecf/config-1.4.2.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.7.10/d2abf9e77736acc4450dc4a3f707fa2c10f5099d/kotlin-stdlib-1.7.10.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/ch.qos.logback/logback-core/1.2.3/864344400c3d4d92dfeb0a305dc87d953677c03c/logback-core-1.2.3.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-core-jvm/1.6.4/2c997cd1c0ef33f3e751d3831929aeff1390cb30/kotlinx-coroutines-core-jvm-1.6.4.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-sessions-jvm/2.1.0/aea94045c1ebe7a7c0218bbcb694d9ff9a9ae12c/ktor-server-sessions-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-client-core-jvm/2.1.0/9f817ae55e463f27d885ef051ff88e4c2acea7da/ktor-client-core-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-host-common-jvm/2.1.0/4f8a7ed4d46d72f78d7eb806d31863e7e6a99626/ktor-server-host-common-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http/4.1.78.Final/b636591e973418479f2d08e881b12be61845aa6f/netty-codec-http-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler/4.1.78.Final/7e902b6018378bb700ec9364b1d0fba6eefd99fd/netty-handler-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.78.Final/e8486c8923fc0914df4562a8e0923462e885f88a/netty-codec-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.78.Final/b1639d431e43622d6cbfdd45c30d3fb810fa9101/netty-transport-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.78.Final/6c0c2d805a2c8ea30223677d8219235f9ec14c38/netty-buffer-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.78.Final/d6f560098050f88ba11750aa856edd955e4a7707/netty-common-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-classes-kqueue/4.1.78.Final/5536de8030ef84507504b352e54c298b9db361c9/netty-transport-classes-kqueue-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-unix-common/4.1.78.Final/62c64e182a11e31ad80e68a94f0b02b45344df23/netty-transport-native-unix-common-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-classes-epoll/4.1.78.Final/2151893a521c6362858071053fddbd270c1f20bd/netty-transport-classes-epoll-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-serialization-jvm/2.1.0/e1d36c91035d2eddd1f202411248a4b6944ea387/ktor-serialization-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-events-jvm/2.1.0/ef58c3c3d012ab115011af1e2f2afc357433c220/ktor-events-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-http-jvm/2.1.0/eff494019c2c1a0e9db6e13533d241ecd6bbc822/ktor-http-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-utils-jvm/2.1.0/1b3ca53f817d054ba00c8a3e92a50ba655428c71/ktor-utils-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-serialization-kotlinx-jvm/2.1.0/c852a1ac0196c9b3e9085b7d19e515f3774025f6/ktor-serialization-kotlinx-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-serialization-json-jvm/1.3.3/6298c404c1159685fb657f15fa5b78630da58c96/kotlinx-serialization-json-jvm-1.3.3.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains/annotations/13.0/919f0dfe192fb4e063e7dacadee7f8bb9a2672a9/annotations-13.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.78.Final/691170d70a979757d50f60e16121ced5f411a9b8/netty-resolver-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-websocket-serialization-jvm/2.1.0/d202979642d5091a18587b3f81269c0a9a8acc78/ktor-websocket-serialization-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-websockets-jvm/2.1.0/17015d23235fd5e784230875e630f6bd1418fdf8/ktor-websockets-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-http-cio-jvm/2.1.0/81fc128a33b4cdf20ec48462d165d071f366397e/ktor-http-cio-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-io-jvm/2.1.0/ccd958db554677538c21515a0acd62862bb6b2a0/ktor-io-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-serialization-core-jvm/1.3.3/3c22103b5d8e3c262db19e85d90405ca78b49efd/kotlinx-serialization-core-jvm-1.3.3.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-network-jvm/2.1.0/e1a24aa8ef91988a7d33ba95b51f21b3cfc334a4/ktor-network-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.fusesource.jansi/jansi/2.4.0/321c614f85f1dea6bb08c1817c60d53b7f3552fd/jansi-2.4.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.13.2.2/ffeb635597d093509f33e1e94274d14be610f933/jackson-databind-2.13.2.2.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.15/49d94806b6e3dc933dacbd8acb0fdbab8ebd1e5d/commons-codec-1.15.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/30.0-jre/8ddbc8769f73309fe09b54c5951163f10b0d89fa/guava-30.0-jre.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.13.2/ec18851f1976d5b810ae1a5fcc32520d2d38f77a/jackson-annotations-2.13.2.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.13.2/a6a0e0620d51833feffc67bccb51937b2345763/jackson-core-2.13.2.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.google.guava/failureaccess/1.0.1/1dcf1de382a0bf95a3d8b0849546c88bac1292c9/failureaccess-1.0.1.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.google.guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/b421526c5f297295adef1c886e5246c39d4ac629/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/3.5.0/2f50520c8abea66fbd8d26e481d3aef5c673b510/checker-qual-3.5.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.4/dac170e4594de319655ffb62f41cbd6dbb5e601e/error_prone_annotations-2.3.4.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-slf4j/1.6.4/9369fcabacb66a55ca3b7939b72bb0b62af112ce/kotlinx-coroutines-slf4j-1.6.4.jar com.example.ApplicationKt
2022-08-12 10:28:22.860 [main] INFO  ktor.application - Autoreload is disabled because the development mode is off.
Exception in thread "main" io.ktor.server.config.ApplicationConfigurationException: Property jwt.audience not found.
	at io.ktor.server.config.MapApplicationConfig.property(MapApplicationConfig.kt:50)
	at com.example.plugins.SecurityKt$configureSecurity$1$1.invoke(Security.kt:17)
	at com.example.plugins.SecurityKt$configureSecurity$1$1.invoke(Security.kt:16)
	at io.ktor.server.auth.jwt.JWTAuthKt.jwt(JWTAuth.kt:326)
	at io.ktor.server.auth.jwt.JWTAuthKt.jwt$default(JWTAuth.kt:322)
	at com.example.plugins.SecurityKt$configureSecurity$1.invoke(Security.kt:16)
	at com.example.plugins.SecurityKt$configureSecurity$1.invoke(Security.kt:15)
	at io.ktor.server.auth.Authentication$Companion.install(Authentication.kt:98)
	at io.ktor.server.auth.Authentication$Companion.install(Authentication.kt:94)
	at io.ktor.server.application.ApplicationPluginKt.install(ApplicationPlugin.kt:98)
	at io.ktor.server.auth.AuthenticationKt.authentication(Authentication.kt:122)
	at com.example.plugins.SecurityKt.configureSecurity(Security.kt:15)
	at com.example.ApplicationKt$main$1.invoke(Application.kt:10)
	at com.example.ApplicationKt$main$1.invoke(Application.kt:8)
	at io.ktor.server.engine.internal.CallableUtilsKt.executeModuleFunction(CallableUtils.kt:51)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$launchModuleByName$1.invoke(ApplicationEngineEnvironmentReloading.kt:334)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$launchModuleByName$1.invoke(ApplicationEngineEnvironmentReloading.kt:333)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartupFor(ApplicationEngineEnvironmentReloading.kt:358)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.launchModuleByName(ApplicationEngineEnvironmentReloading.kt:333)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.access$launchModuleByName(ApplicationEngineEnvironmentReloading.kt:32)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:321)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:312)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartup(ApplicationEngineEnvironmentReloading.kt:340)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.instantiateAndConfigureApplication(ApplicationEngineEnvironmentReloading.kt:312)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createApplication(ApplicationEngineEnvironmentReloading.kt:149)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.start(ApplicationEngineEnvironmentReloading.kt:279)
	at io.ktor.server.netty.NettyApplicationEngine.start(NettyApplicationEngine.kt:210)
	at com.example.ApplicationKt.main(Application.kt:12)
	at com.example.ApplicationKt.main(Application.kt)

Process finished with exit code 1

To reproduce, write the following code:

import io.ktor.server.application.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import io.ktor.server.response.*
import io.ktor.server.routing.*

fun main(args: Array<String>) {
    val x = args.firstOrNull()

    embeddedServer(Netty, port = 8888) {
        routing {
            get("/") {
                call.respondText { x.toString() }
            }
        }
    }.start(wait = true)
}

And then apply the Extract Application Module from the current emdeddedServer block quick fix.
As a result, the generated code isn't compilable:

fun main(args: Array<String>) {
    val x = args.firstOrNull()

    embeddedServer(Netty, port = 8888, module = Application::myApplicationModule).start(wait = true)
}

fun Application.myApplicationModule() {
    routing {
        get("/") {
            call.respondText { x.toString() } // Unresolved reference: x
        }
    }
}

call.respondRedirect("https://www.google.com")

Note: in Ktor, this function is implemented as following:

public suspend fun ApplicationCall.respondRedirect(url: String, permanent: Boolean = false) {
    response.headers.append(HttpHeaders.Location, url)
    respond(if (permanent) HttpStatusCode.MovedPermanently else HttpStatusCode.Found)
}

Expected: if permanent = false, HttpStatusCode.MovedPermanently should be the status code, and HttpStatusCode.Found otherwise.
Actual: no information in OpenAPI file

While it works fine in main module, if you add a separate YAML config for tests, that's what you'll see:

intall(StatusPages) {
  exception<E> { ... }
  unhandledException()
}

fun StatusPagesConfig.unhandledException() {
  call.respond(...)
}

Here, call.respond from unhandledException must be present in OpenAPI as well.

Example:

get("/a/b/c") {
  call.respond(randomCoordinate())
  exampleRespond(Coordinate(10, 20))
}

class Coordinate(val x: Int, val y: Int)

fun randomCoordinate(): Coordinate = Coordinate(Random.nextInt(), Random.nextInt())

fun Route.exampleRespond(value: Any?) {
  // A hack to add OpenAPI examples without affection actuall HTTP behaviour:
  if (false) { 
    call.respond(value)
  }
}

Expected: OpenAPI shows that /a/b/c route has #ref/Coordinate response type reference.
(Ideally, it also has to show value: x: 10 y: 20 as an example value for that response type, but it is not supported yet (KTOR-5683))
Actual: OpenAPI shows that /a/b/c route has an object response type.

Example: Open Qodana Cloud project and generate OpenAPI for core-api module.
Computation will run for around 1 minute without an option to cancel.

Example:

get("/a/b/c") {
  call.respond(Coordinate(10, 20))
}

class Coordinate(val x: Int, val y: Int)

Expected: OpenAPI has to show value: x: 10 y: 20 as an example value for that response type of /a/b/c request.
Actual: example it is not reflected in OpenAPI.

Example:

call.respond(MyData(x = 1, y = 2, items = listOf("a", "b", "c")))

Or even:

call.respond(MyData(x = 1, y = 2))
...
class MyData(val x: Int, val y: Int, val items: List<String> = listOf("a", "b", "c"))

OpenAPI should have an example entrance:

examples:
  Example#1:
    description: ""
    value:
      x: 1
      y: 1
      items:
        - "a"
        - "b"
        - "c"

Example:

routing {
    get("/my/route") {
        sendValue(MyObject(1, 2))
    }
}
...
fun  <T>  PipelineContext<Unit, ApplicationCall>.sendValue(value: T) {
    call.respond(HttpStatusCode.Accepted, value)
}

Expected: After generating OpenAPI documentation, it should tell about MyObject(1, 2) response with example value based on what was passed to sendValue function.

Example:

data class PaginatedList<T>(
    @Expose val count: Long,
    @Expose val next: Long?,
    @Expose val prev: Long?,
    @Expose val items: List<T>,
)

data class ProjectListResponseItem(
    @Expose val id: String,
    @Expose val name: String,
    @Expose val slug: String = name.asSlug,
    @Expose val branch: String? = null,
    @Expose val commit: String? = null,
    @Expose val lastChecked: Instant? = null,
    @Expose @NoSlug val lastReportId: String? = null,
    @Expose val lastState: ReportState?,
    @Expose val url: String? = null,
    @Expose val problems: Problems? = null,
    @Expose val baselineCount: Int? = null,
)
...
call.respond(results())
...
fun results(): PaginatedList<ProjectListResponseItem>

Expected:

responses:
        "200":
          description: ""
          content:
            '*/*':
              schema:
                $ref: "#/components/schemas/PaginatedList_ProjectListResponseItem"
...
schemas:
    PaginatedList_ProjectListResponseItem:
      type: "object"
      properties:
        count:
          type: "integer"
          format: "int64"
        next:
          type: "integer"
          format: "int64"
        prev:
          type: "integer"
          format: "int64"
        items:
          type: "array"
          items:
            $ref: "#/components/schemas/ProjectListResponseItem"
    ProjectListResponseItem:
      type: "object"
      properties:
        id:
          type: "string"
        name:
          type: "string"
        slug:
          type: "string"
        branch:
          type: "string"
        commit:
          type: "string"
        lastChecked:
          $ref: "#/components/schemas/Instant"
        lastReportId:
          type: "string"
        lastState:
          type: "string"
          enum:
          - "UPLOADING"
          - "UPLOADED"
          - "PROCESSED"
          - "FAILED_TO_PROCESS"
          - "PINNED"
          - "MARKED_FOR_DELETION"
          - "OVERWRITTEN"
        url:
          type: "string"
        problems:
          $ref: "#/components/schemas/Problems"
        baselineCount:
          type: "integer"
          format: "int32"

Example:
There is a StatusPages plugin that handles exceptions:

exception<MyExceptionType> { call, cause ->
  val descriptor = ApiDescr(cause.name, cause.message)
  call.respond(cause.statusCode, descriptor) // <--- HERE
}

And routes:

get("/first/route") {
   throw MyExceptionType1("first name", "Cool message!")
}

get("/second/route") {
   throw MyExceptionType2("2nd name", "Bad message :(")
}

And exceptions:

class MyExceptionType(val statusCode: Int, val type: String, val details: String)
class MyExceptionType1:(type: String, details: String): MyExceptionType(HttpStatusCode.BAD_REQUEST, type, details)
class MyExceptionType2:(type: String, details: String): MyExceptionType(HttpStatusCode.UNAUTHORIZED, type, details)

Expected OpenAPI:

/first/route:
    get:
      description: ""
      responses:
        "400":
          description: "Bad Request"
          content:
            '*/*':
              schema:
                $ref: "#/components/schemas/ApiDescr"
              examples:
                Example#1:
                  value:
                    type: "first name"
                    details: "Cool message!"
/second/route:
    get:
      description: ""
      responses:
        "401":
          description: "Unauthorized"
          content:
            '*/*':
              schema:
                $ref: "#/components/schemas/ApiDescr"
              examples:
                Example#1:
                  value:
                    type: "2nd name"
                    details: "Bad message :("
components:
  schemas:
    APIError:
      type: "object"
      properties:
        type:
          name: "string"
        details:
          message: "string"

Actual OpenAPI: nothing is known about response types at all.

Note: even if status codes are known statically (written in exception<...> handlers), the APIError messages can not be deduced from exceptions.

Example:

ktor:
  application:
    modules:
      - a.b.c.myModule1
      - a.b.c.myModule2

fun Application.myModule1() {
 // routes
}

fun Application.myModule2() {
 // status pages
}

Expected: In this case, all exceptions thrown from myModule1 must be handled by StatusPages defined in myModule2. And this must be represented in OpenAPI.
Actual: OpenAPI generator does not know that myModule2 is also included to the application configuration and hence it does not analyze StatusPages from there because it is not directly referenced.

OpenAPI generator has to check application configuration files to know which other modules it has to analyze.

Example:

fun Application.main() {
  routing {
    f()
  }
}

fun Route.f() {
  get("/f") {...}
  g()
}

fun Route.g() {
  route("/g") {
    f()
  }
}

Expected: at least routes /f and g/f are shown.
Actual: no routes are shown, StackOverflow is thrown.

Example:

package a.b.c

import io.ktor.server.Application

fun Application.myModule() { ... }

yaml:

ktor:
  application:
    modules:
      - a.b.c.myModule

Expected: a.b.c.myModule is a reference in yaml leading to Application.myModule().
Actual: it does not resolve.

Example:

1. Without default values:

ktor:
  deployment:
    port: $PORT

2. And with default values:

ktor:
  deployment:
    port: "$PORT:8080"

Expected highlighting:


{width=70%}

Actual: currently, the whole "$PORT:8080" is seen as one single string that is a candidate to be converted to Integer, because variables are not supported.

Searching for "Content Negotiation" both in the web and IntelliJ wizards gives the following results:

I suppose this is because ContentNegotiation doesn't contain a space (even though other features do).
I suggest:

• Renaming the feature to Content Negotiation
• Ignoring whitespace in the fuzzy search.

Documentation for the Endpoints tool window says that WebSocket endpoints are supported:

The Endpoints tool window provides an aggregated view of both client and server APIs used in your project for HTTP and WebSocket protocols.

It would be useful to have this capability for Ktor projects. Currently, only HTTP routes are displayed.

io.ktor.request.RequestAlreadyConsumedException: Request body has been already consumed (received).
at io.ktor.features.DoubleReceive$Feature$install$1.invokeSuspend(DoubleReceive.kt:94)

embeddedServer(Jetty, 8080) {
       install(ContentNegotiation) {
           json(json = json)
        }
        install(DoubleReceive) {
            this.receiveEntireContent = true
        }
        install(CallLogging) {
            format {
                runBlocking {
                    it.receiveText()
                }
            }

        }
    }

I was surprised that kotlinx.coroutines Flow<T> is currently not supported by ktor-serialization, Flow is not collected so nothing happens.

What steps will reproduce the issue?

This is a minimal reproducer

fun main(args: Array<String>) {
    embeddedServer(Netty, 8080) {
        install(ContentNegotiation) {
            json()
        }
        routing {
            get("/") {
                call.respond(
                    flow {
                        emit(1)
                        delay(1000)
                        emit(2)
                    }
                )
            }
        }
    }.start(wait = true)
}

What is expected
As coroutines are heavilly used and supported in Ktor, I think ktor-serialization should support kotlinx.coroutines Flow<T> natively, as it is the official asynchronous Stream on Kotlin coroutines world.
I don't think that the Flow<T> support should require an additional Plugin, but you can of course disagre with my opinion ;)

The minimal support (on multiplatform) can be to just collect the Flow<T> as a List<T>, and then serialize it.

Sequence<T> should be available for deserialization too.

Bonus 1:

On JVM (with kotlinx.serialization, Jackson and Gson) normal and Flow<T> serialization can be optimized, and asynchronously write in the OutpuStream (server mode) and read from InpuStream (client mode), or using java.io.Writer/java.io.Reader. Examples are available here in a kotlinx.serialization JsonLazySequenceTest.kt
I made a custom implementation to support Flow in Ktor server mode in this class on my kotysa-ktor-r2dbc-coroutines sample project -> by the way if you have a few minutes to read this sample's very short code, and tell me if I made a mistake thanks a lot ^^

Bonus 2:

Flow<T> could also be used to asynchronously serialize data as Server Sent Events and in Websocket in ktor-websocket-serialization.

The unofficial application/x-json-stream (MediaType.APPLICATION_JSON_STREAM) could be supported aswell, even if not supported by Browsers and non standard yet, this format can be useful for server-to-server JSON communication. See this micronaut topic

https://github.com/ktorio/ktor/issues/514

/**
 * Support pre-compressed files in the file system only (not just any classpath resource)
 * ...
public fun Route.preCompressed( 

It looks like preCompressed only works with files() at the moment, why ?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/119

We have continue support in netty however it shouldn't be done like that: we have to respond continue (to allow client sending request body) only when handler tries to access request body (call.request.receive()) or at least once request handler has started (that is not exactly correct as user's code could respond 404)

To reproduce run the following test:

class WebsocketsTest {
    private val server = embeddedServer(CIO, host = "127.0.0.1", port = 0) {
        install(WebSockets)

        routing {
            webSocket("/") {
                for (frame in incoming) {
                    outgoing.send(frame)
                }
            }
        }
    }

    private val client = HttpClient(io.ktor.client.engine.cio.CIO) {
        install(io.ktor.client.plugins.websocket.WebSockets)
    }

    @Before
    fun setup() {
        server.start(wait = false)
    }

    @After
    fun dispose() {
        server.stop()
    }

    @Test
    fun test() = testSuspend {
        val session = createSession()
        session.outgoing.send(Frame.Ping(ByteArray(126)))
        val frame = session.incoming.receive()

        assertIs<Frame.Close>(frame)
        assertEquals(1002, frame.readReason()?.code)
    }

    private suspend fun createSession(): ClientWebSocketSession {
        val url = "ws://127.0.0.1:${server.resolvedConnectors().first().port}/"
        return client.webSocketRawSession { url(url) }
    }
}

To reproduce run the following test:

class WebsocketsTest {
    private val server = embeddedServer(CIO, host = "127.0.0.1", port = 0) {
        install(WebSockets)

        routing {
            webSocket("/") {
                for (frame in incoming) {
                    outgoing.send(frame)
                }
            }
        }
    }

    private val client = HttpClient(io.ktor.client.engine.cio.CIO) {
        install(io.ktor.client.plugins.websocket.WebSockets)
    }

    @Before
    fun setup() {
        server.start(wait = false)
    }

    @After
    fun dispose() {
        server.stop()
    }

    @Test
    fun test() = testSuspend {
        val session = createSession()
        session.outgoing.send(Frame.Text(false, "hello ".toByteArray()))
        session.outgoing.send(Frame.Text(true, "world".toByteArray()))
        
        val frame = session.incoming.receive()
        assertIs<Frame.Close>(frame)
    }

    private suspend fun createSession(): ClientWebSocketSession {
        val url = "ws://127.0.0.1:${server.resolvedConnectors().first().port}/"
        return client.webSocketRawSession { url(url) }
    }
}

When providing files via a static block, a non-standard content-type is used.

Code:

fun Application.main() {
    install(Routing)
    
    static("assets") {
        files("public")
    }
}

File structure:

\- public
  |- some-html-file.html
  |- some-image.png
  \- some-ttf-font.ttf

Making requests to the following urls returns a response with the expected Content-Type header:

• /assets/some-html-file.html returns Content-Type: text/hml
• /assets/some-image.png returns Content-Type: image/png

However, making a request to /assets/some-ttf-font.ttf returns an unexpected Content-Type.

Example response:

HTTP/1.1 200 OK
Date: Thu, 08 Dec 2022 21:23:03 GMT
Server: Ktor/2.0.3
Last-Modified: Mon, 15 Aug 2022 22:09:01 GMT
Accept-Ranges: bytes
Content-Length: 335196
Content-Type: application/x-font-ttf

According to the IANA, the correct MIME type for a .ttf file is font/ttf.

Here is a list of file types which return an incorrect Content-Type:

• ttf
  • Expected: font/ttf
  • Actual: application/x-font-ttf
• woff2
  • Expected: font/woff2
  • Actual: application/octet-stream
• woff
  • Expected: font/woff
  • Actual: application/x-font-woff
• otf
  • Expected: font/otf
  • Actual: application/x-font-otf
• gz
  • Expected: application/gzip
  • Actual: application/x-compressed

The source of the issues is the Mimes.kt file in ktor-http module.

I want to avoid logging static file requests，currently I avoid logging static file requests like this:

  install(CallLogging) {
      level = Level.INFO
      filter { !it.parameters.contains("static-content-path-parameter") }
  }

Since the pathParameterName property of StaticContent.kt is private, I have to manually copy the String. Maybe we can make the pathParameterName String property of StaticContent.kt public ，or add configuration items in the CallLogging plugin.

https://jbs.zendesk.com/agent/tickets/4182169

My Environment

iosX64 Native - iOS 16.0 Simulator
Kotlin: 1.8.0
XCode: 14.0.1
Ktor Version: 2.2.3

Background

I'm running a Ktor server for mocking purposes on an iOS simulator. The problem is that when killing the application through XCode, I get no callback to stop the server gracefully (i.e. it's a SIGKILL).

If a request was in progress when the app is killed, it leads to the following exception upon restarting the app (unless I wait about 30 seconds).

io.ktor.utils.io.errors.PosixException.AddressAlreadyInUseException: EADDRINUSE

Feature Request

(I'm a mobile engineer so apologies if what I'm asking is incorrect). Can there be a flag added to the embeddedServer to allow port reuse, so that killing and restarting the app with the same server port does not lead to a crash?

I.e exposing the API used here to the embeddedServer configuration: https://youtrack.jetbrains.com/issue/KTOR-4442/EADDRINUSE-on-socket-bind-on-Linux.

Minimal Reproducible Example

https://github.com/SamCosta1/ktor-connection-bug-example/tree/ktor-killing-app-example

Notes

I've not experienced this issue running on Android, this seems to be Native only.

We would like to provide a solution for loading multiple configuration files, i.e. java -jar sample-app.jar -config=application.conf -config=applicationFoo.conf -config=applicationBar.conf.

The mechanism should work from left to right - it means if application.conf and applicationFoo.conf have common properties then values from application.conf should be overwritten by values from applicationFoo.conf, like shown below:

application.conf

ktor {
    deployment {
        port = 8080
    }

    application {
        class = <org.company.ApplicationClass>
    }
}

applicationFoo.conf

ktor {
    deployment {
        port = 8085
        testProperty = "TestValue"
    }
}

config {
    configurationProperty = "Config value"
}

result configuration:

ktor {
    deployment {
        port = 8085
        testProperty = "TestValue"
    }
    
    application {
        class = <org.company.ApplicationClass>
    }
}

config {
    configurationProperty = "Config value"
}

This used to work in 2.2.3:

fun main() {
  embeddedServer(Netty, port = 8082, host = "0.0.0.0", module = Application::module)
    .start(wait = true)
}

fun Application.module() {
  install(ContentNegotiation) {
    json()
  }
  routing {
    get {
      call.respond(value())
    }
  }
}

// fun value(): Data = Data("bar") works
fun value(): Any = Data("bar")

@Serializable
data class Data(val foo: String)

on 2.2.4 ktor throws

2023-03-21 01:45:13.069 [eventLoopGroupProxy-4-1] ERROR ktor.application - Unhandled: GET - /
kotlinx.serialization.SerializationException: Serializer for class 'Any' is not found.
Mark the class as @Serializable or provide the serializer explicitly.

so I have a simplest ktor app

fun main() {
    val env = applicationEngineEnvironment {
        module {
            routing {
                get("/") {
                    call.respondText("Hello, world!")
                }
            }
        }

        connector {
            this.port = 8080
        }
    }
    embeddedServer(CIO, env).start(wait = true)
}

    implementation("io.ktor:ktor-server-core:2.2.3")
    implementation("io.ktor:ktor-server-cio:2.2.3")

if you perform a scan of open tcp ports like this nmap localhost -Pn -p8080, the app crashes with

Exception in thread "DefaultDispatcher-worker-2" java.net.SocketException: Invalid argument
        at java.base/sun.nio.ch.Net.setIntOption0(Native Method)
        at java.base/sun.nio.ch.Net.setSocketOption(Net.java:455)
        at java.base/sun.nio.ch.Net.setSocketOption(Net.java:393)
        at java.base/sun.nio.ch.SocketChannelImpl.setOption(SocketChannelImpl.java:280)
        at io.ktor.network.sockets.ServerSocketImpl.accepted(ServerSocketImpl.kt:51)
        at io.ktor.network.sockets.ServerSocketImpl.acceptSuspend(ServerSocketImpl.kt:42)
        at io.ktor.network.sockets.ServerSocketImpl.access$acceptSuspend(ServerSocketImpl.kt:12)
        at io.ktor.network.sockets.ServerSocketImpl$acceptSuspend$1.invokeSuspend(ServerSocketImpl.kt)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
        Suppressed: kotlinx.coroutines.DiagnosticCoroutineContextException: [LazyStandaloneCoroutine{Cancelling}@23a3bd4f, Dispatchers.IO]

Hi,

I am trying to integrate with a service using OAuth2, however the response that the service gives does not get handled correctly by ktor. From the Asana website, this is a typical response given (https://developers.asana.com/docs/oauth)

{
  "access_token": "f6ds7fdsa69ags7ag9sd5a",
  "expires_in": 3600,
  "token_type": "bearer",
  "data": {
    "id": 4673218951,
    "gid": "4673218951",
    "name": "Greg Sanchez",
    "email": "gsanchez@example.com"
  },
  "refresh_token": "hjkl325hjkl4325hj4kl32fjds"
}

Currently ktor expects the response to only contain primitives. When trying to get the response after exchanging the code for the token, I get this error message from ktor.

Element class kotlinx.serialization.json.JsonObject is not a JsonPrimitive

I was wondering if this could be supported, possibly by adding these as extra parameters

Thanks!

Ktor version: 2.2.4

Kotlin version: 1.8.10

When running ktor server like:

val server = embeddedServer(Netty, port = 28080) {
        install(CallLogging) {
            callIdMdc()
        }
        routing {
            get("/test") {
                this.call.respondText("Hello, world!")
            }
            get("/err") {
                throw Exception("Unexpected")
            }
        }
    }

It will not log calls to /err route:

    @Test
    fun testRoot() = runTest {
        server.start(false)

        val client = HttpClient(CIO)
        assertEquals(200, client.get("http://localhost:28080/test").status.value)
        assertEquals(500, client.get("http://localhost:28080/err").status.value)

        client.close()
        server.stop() 
   }

Produced logs:

2023-03-16 10:14:44,668 INFO [] Application started in 0.284 seconds.                                            [ktor.application] [oroutine#2]
2023-03-16 10:14:44,669 INFO [] Application started: io.ktor.server.application.Application@4f3356c0             [ktor.application] [oroutine#2]
2023-03-16 10:14:45,043 INFO [] Responding at http://0.0.0.0:28080                                               [ktor.application] [oroutine#1]
2023-03-16 10:14:45,641 INFO [] 200 OK: GET - /test in 163ms                                     [ktor.application] [handler#16]
2023-03-16 10:14:45,669 ERROR [] Unhandled: GET - /err                                                            [ktor.application] [handler#29]
java.lang.Exception: Unexpected
	(Coroutine boundary)
	at io.ktor.util.pipeline.PipelineKt$execute$2.invokeSuspend(Pipeline.kt:478)
	at io.ktor.server.routing.Routing.executeResult(Routing.kt:188)
2023-03-16 10:14:47,492 INFO [] Application stopping: io.ktor.server.application.Application@4f3356c0            [ktor.application] [oroutine#2]
2023-03-16 10:14:47,493 INFO [] Application stopped: io.ktor.server.application.Application@4f3356c0             [ktor.application] [oroutine#2]

When CallLogin plug is not configured with mds, it will produces logs:

2023-03-16 10:13:47,070 INFO [] Application started in 0.269 seconds.                                            [ktor.application] [oroutine#2]
2023-03-16 10:13:47,071 INFO [] Application started: io.ktor.server.application.Application@6ac45c0c             [ktor.application] [oroutine#2]
2023-03-16 10:13:47,432 INFO [] Responding at http://0.0.0.0:28080                                               [ktor.application] [oroutine#1]
2023-03-16 10:13:48,031 ERROR [] Unhandled: GET - /err                                                            [ktor.application] [handler#29]
java.lang.Exception: Unexpected
	(Coroutine boundary)
	at io.ktor.util.pipeline.PipelineKt$execute$2.invokeSuspend(Pipeline.kt:478)
	at io.ktor.server.routing.Routing.executeResult(Routing.kt:188)
2023-03-16 10:13:48,170 INFO [] 200 OK: GET - /test in 312ms                                     [ktor.application] [handler#16]
2023-03-16 10:13:48,717 INFO [] 500 Internal Server Error: GET - /err in 730ms                   [ktor.application] [handler#29]
2023-03-16 10:13:49,761 INFO [] Application stopping: io.ktor.server.application.Application@6ac45c0c            [ktor.application] [oroutine#2]
2023-03-16 10:13:49,762 INFO [] Application stopped: io.ktor.server.application.Application@6ac45c0c             [ktor.application] [oroutine#2]

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/233

I want Ktor to drive my website. I have a bunch of HTML files but I want to strip the HTML suffix from those files when they are being served as the HTML suffix is just a technical detail that I want to strip.

When serving static files in static routes using the files(dir) DSL, it would be nice if a HEAD mapping was setup in addition to the GET one, in order to respect HTTP semantics.

The HEAD mapping should return (among others) the correct status code (e.g. 404 if the file doesn't exist) and also the file size as Content-Length for instance.

I stumbled upon this issue when trying to serve my local .m2 directory as static files to emulate a remote server, and Ivy (and likely Maven and Gradle) makes use of HEAD requests.

I had this simple route, serving the content of the assets resource folder when getting assets/*

static("assets") {
    resources("assets")
}

I realized that Ktor was responding to these requests with a weird Content-Type for files ending in .map like *.js.map (or *.css.map I guess).
This doesn't cause any real issue as most clients getting this kind of files ignore content type entirely. Still, I wanted to change that content type to something a little more standard like application/octet-stream or even something less standard but more useful for debugging, application/json.

So I tried to change this content type and couldn't find a way. I tried with an interceptor but you're somehow not allowed to change the content type header.
I ended up having to drop the use of static/resources. So now I have this:

    val oneYearInSeconds = Duration.ofDays(365).toSeconds()
    val pathParameterName = "path"
    get("/assets/{$pathParameterName...}") {
        val relativePath = call.parameters
            .getAll(pathParameterName)
            ?.joinToString(File.separator)
            ?: return@get

        val content = call.resolveResource(relativePath, "assets") { extension ->
            when {
                extension.endsWith(".js.map") -> ContentType.Application.Json
                else -> ContentType.defaultForFileExtension(extension)
            }
        } ?: return@get

        call.response.header(HttpHeaders.CacheControl, "public, max-age=$oneYearInSeconds, immutable")
        call.respond(content)
    }

Which is kinda longer and unfortunately doesn't use what's already there for static resources.

(I also implemented caching there, because 1. there does not seem to be caching enabled by default for static content and 2. Ktor's caching headers feature kinda sucks, but that's a different story)

Using the code found in the micrometer-metrics code snippet, and adding the dependencies for the dropwizard-metrics, if the plugin installation block for the Dropwizard plugin is added after the MicrometerMetrics plugin:

    install(MicrometerMetrics) {
        registry = PrometheusMeterRegistry(PrometheusConfig.DEFAULT)
        distributionStatisticConfig = DistributionStatisticConfig.Builder()
            .percentilesHistogram(true)
            .maximumExpectedValue(Duration.ofSeconds(20).toNanos().toDouble())
            .serviceLevelObjectives(
                Duration.ofMillis(100).toNanos().toDouble(),
                Duration.ofMillis(500).toNanos().toDouble()
            )
            .build()
        meterBinders = listOf(
            ClassLoaderMetrics(),
            JvmMemoryMetrics(),
            JvmGcMetrics(),
            ProcessorMetrics(),
            JvmThreadMetrics(),
            FileDescriptorMetrics(),
            UptimeMetrics()
        )
    }
    install(DropwizardMetrics) {
        JmxReporter.forRegistry(registry)
            .convertRatesTo(TimeUnit.SECONDS)
            .convertDurationsTo(TimeUnit.MILLISECONDS)
            .build()
            .start()
    }

then when the /metrics endpoint is scrapped, an exception is thrown;

java.lang.ClassCastException: class io.ktor.server.metrics.dropwizard.CallMeasure cannot be cast to class io.ktor.server.metrics.micrometer.CallMeasure (io.ktor.server.metrics.dropwizard.CallMeasure and io.ktor.server.metrics.micrometer.CallMeasure are in unnamed module of loader 'app')
        at io.ktor.server.metrics.micrometer.MicrometerMetricsKt$MicrometerMetrics$2$4.invoke(MicrometerMetrics.kt:148)
        at io.ktor.server.metrics.micrometer.MicrometerMetricsKt$MicrometerMetrics$2$4.invoke(MicrometerMetrics.kt:146)
        at io.ktor.server.application.hooks.ResponseSent$install$1.invokeSuspend(CommonHooks.kt:110)
        at io.ktor.server.application.hooks.ResponseSent$install$1.invoke(CommonHooks.kt)
        at io.ktor.server.application.hooks.ResponseSent$install$1.invoke(CommonHooks.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(SuspendFunctionGun.kt:88)
        at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invokeSuspend(DefaultTransform.kt:29)
        at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
        at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
        at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
        at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
        at io.ktor.server.engine.DefaultEnginePipelineKt.tryRespondError(DefaultEnginePipeline.kt:122)
        at io.ktor.server.engine.DefaultEnginePipelineKt.handleFailure(DefaultEnginePipeline.kt:54)
        at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invokeSuspend(DefaultEnginePipeline.kt:40)

A workaround appears to be registering the Dropwizard plugin before the MicrometerMetrics plugin

It is now not possible to serve static files from resources in GraalVM native image with default configuration:

routing {
    static("/assets") {
        resources("assets")
    }
}

It is because internal fun resourceClasspathResource supports only file, jar and jrt protocols.

But in native image, resources are supported by resource protocol.

PR: https://github.com/ktorio/ktor/pull/3388

Module: ktor-server-auth-jvm-2.2.2.jar
Class: io/ktor/server/auth/SessionAuth.kt:123

This line of code does not work for unauthorized users:
session<UserSession>("auth-session")

Since the default challengeFunction parameter is an empty implementation, users without a Session can also access resources protected by auth-session

{width=70%}

    install(Authentication) {
        form("auth-form") {
            validate { credentials ->
                if (credentials.name == "123" && credentials.password == "123") {
                    UserIdPrincipal(credentials.name)
                } else {
                    null
                }
            }
        }
        session<UserSession>("auth-session") // not work
    }

Steps to reproduce: serve static JS module files from a folder in a standard way like

routing {
    trace { application.log.trace(it.buildText()) }

    static("/scripts/") {
        this.staticRootFolder = File(rootPath)
        files(".")
    }
}

The problem: the MIME type for .mjs files is wrong and the browser cannot load the modules.

Error in the browser console:

Loading module from “http://localhost:81/scripts/asdf.mjs” was blocked because of a disallowed MIME type (“application/octet-stream”).

Expected result: ContentType header should be "text/javascript"

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1781

Subsystem
Server, route building

Is your feature request related to a problem? Please describe.
Right now the accept() route selector only takes a single content type.

Describe the solution you'd like
It would be convenient if we could pass more than one, and it could match on any of the ones provided.

Motivation to include to ktor
That would reduce the amount of duplication when multiple content types are handled by the same handler.

If I install(ContentNegotiation) { gson() } and install(Mustache)
call.respond(MustacheContent("temp.hbs", mapOf("user" to sampleUser))) stops working correctly

Instead of returning proper html output, the server will return a json

{"template":"temp.hbs","model":{"user":{"id":1,"name":"John"}},"contentType":{"contentType":"text","contentSubtype":"html","content":"text/html","parameters":[{"name":"charset","value":"UTF-8","escapeValue":false}]}}

the return content type is application/json

After removing install(ContentNegotiation) { gson() } block, html is responded correctly but other api routes throw serialisation related errors as expected due to lack of gson

To reproduce send a text Message fragmented into 2 fragments, then Continuation Frame with FIN = false where there is nothing to continue, then an unfragmented Text Message, all sent in one chop.

Expected behavior: the connection is failed immediately since there is no message to continue.
Actual behavior: two text frames are sent back.

embeddedServer(CIO, host = "127.0.0.1", port = 0) {
    install(WebSockets)

    routing {
        webSocket("/") {
            for (frame in incoming) {
                outgoing.send(frame)
            }
        }
    }
}

I have written a simple websocket server with the following configuration:

install(WebSockets) {
    pingPeriod = Duration.ofSeconds(30)
}

Here is an excerpt of the logs:

2023-03-19 17:49:25.020 [DefaultDispatcher-worker-25] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: sending ping frame
2023-03-19 17:49:25.023 [DefaultDispatcher-worker-39] TRACE io.ktor.websocket.WebSocket - WebSocketSession(StandaloneCoroutine{Active}@5a55a4c1) receiving frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:49:25.023 [DefaultDispatcher-worker-39] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: received valid pong frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:49:25.023 [DefaultDispatcher-worker-39] TRACE io.ktor.websocket.WebSocket - WebSocket pinger has timed out
2023-03-19 17:49:55.028 [DefaultDispatcher-worker-5] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: sending ping frame
2023-03-19 17:49:55.029 [DefaultDispatcher-worker-52] TRACE io.ktor.websocket.WebSocket - WebSocketSession(StandaloneCoroutine{Active}@5a55a4c1) receiving frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:49:55.029 [DefaultDispatcher-worker-18] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: received valid pong frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:49:55.029 [DefaultDispatcher-worker-18] TRACE io.ktor.websocket.WebSocket - WebSocket pinger has timed out
2023-03-19 17:50:25.035 [DefaultDispatcher-worker-30] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: sending ping frame
2023-03-19 17:50:25.037 [DefaultDispatcher-worker-15] TRACE io.ktor.websocket.WebSocket - WebSocketSession(StandaloneCoroutine{Active}@5a55a4c1) receiving frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:50:25.038 [DefaultDispatcher-worker-15] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: received valid pong frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:50:25.038 [DefaultDispatcher-worker-15] TRACE io.ktor.websocket.WebSocket - WebSocket pinger has timed out

You can see that each time the server sends a PING frame, it immediately receives a PONG one, but each time we always get the following log:

TRACE io.ktor.websocket.WebSocket - WebSocket pinger has timed out

This is NOT true, the pong frame has been received in time (which is the default here, 15 seconds).

I found this in the code:

LOGGER.trace("WebSocket pinger has timed out")

if (rc == null) {
    // timeout
    // we were unable to send the ping or hadn't got a valid pong message in time,
    // so we are triggering close sequence (if already started then the following close frame could be ignored)

    onTimeout(CloseReason(CloseReason.Codes.INTERNAL_ERROR, "Ping timeout"))
    break
}

Shouldn't we put the log in the condition body? Currently, this is confusing.

TestApplicationEngine has a few runBlocking calls inside it.
This can cause trouble for advanced testing, when a suspend function is tested with a coroutineContext.

One notable example is Kotest. In order to use withClue, coroutineContext should contain errorCollectorContextElement. Otherwise context switching might result in exception. Right now there is no way to pass it into internal runBlocking calls of TestApplicationEngine, and it isn't passed from parent context.

Here's a sample code which reproduces the situation above:

@Test
fun test() = runBlocking(errorCollectorContextElement) {
        testApplication {
            withContext(Dispatchers.Unconfined) {
                withClue("some hint") {
                    withContext(Dispatchers.Unconfined) {
                        delay(2)
                        1 shouldBe 1
                    }
                }
            }
        }
    }

Kotest throws an IndexOutOfBoundsException here, because an error hint is popped from wrong errorCollector (one from another thread).

This particular error can be fixed on Kotest side, but I assume where might be more situations similar to this. Another example might be Exposed framework. One can create a wrapper transaction to rollback it after the test is finished, and context switching might result in losing context with the wrapper transaction.

Design defaults for the static content plugin to make it work out of the box.

https://sbom.lift.sonatype.com/report/T1-a0368c8f29fdaa555824-7e2f1cfcaa55a1-1677578344-d0731128c3504dc0ae64b539fb589112

Please use separate commit for every dependency

routing {
    route(Regex("""/abc/""")) {
        get(Regex("""cde""")) {
            call.respondText { "OK" }
        }
    }
}

None of the /abc/cde and /abc//cde is matched

routing {
    get(Regex("""\(?<a>\)""")) {
        call.respondText { "OK" }
    }
}

On request GET /<a>) there is IllegalArgumentException: No group with name <a>

get(Regex("""\/abc""")) should have the same behavior as get(Regex("""/abc"""))

Subsystem

Client/Server

Motivation

It is much more useful to compare status codes between each other. For example:

httpResponse.status < HttpStatusCode.OK

Currently, in case, if I want to compare status codes I should use next comparison:

httpResponse.status.value < HttpStatusCode.OK.value

Solution

In my PR HttpStatusCode will implements Comparable interface and realize compareTo with any other HttpStatusCode

There are two parameters that are used for shutdown configuration: gracePeriod and shutdownTimeout`. There are used in ShutdownHook and stop method

In KTOR-5359 configuration for ShutdownHook was added

Multiple problems related to private responses caching:

• private cache responses can be shared between multiple users when using HttpClient in a proxy use case
• s-maxage had a typo and was used for private responses, while should be used for shared
• only first Cache-Control header was checked, but they should be merged