We have an API that generates a document served by AWS cloudfront and returns a URL to download the document when it is complete. The problem is that the URL has an extra leading & in the query as in: "?&Expires". When the Ktor client makes the call it drops the leading ampersand which in my mind is correct since it really has no meaning. But unfortunately without the ampersand the call fails.

There is no way to force Ktor not to drop the ampersand.

Cloudfront should not be generating a URL like this and even worse should not be treating the presence as significant and I have a back end person looking into it, but as it stands now we have to have the extra ampersand and will have to hack and expect/actual API to use the platform specific Http Client.

After testing the KTOR-5650 fix (https://github.com/ktorio/ktor/pull/3456) in Electron the check turned out to return false even though we have the Node API. In Electron at this point there can be two process objects and the correct one is defined on window.process, not just process. So we'd have to first detect which process object to use and after that the remaining .versions.node check can be done.

When BearerAuthProvider calls AuthTokenHolder.setToken, it passes a block that is configurable by ktor consumers (unliked BasicAuthProvider and DigestAuthProvider). When this block throws an exception, it results in the skipping of newDeferred.complete(newTokens) and refreshTokensDeferred.value = null, which causes the next call to setToken to hang indefinitely on deferred.await(). Here are the relevant blocks of code:

https://github.com/ktorio/ktor/blob/5f5947ebae390cece2c1a22ac92d5651d764471d/ktor-client/ktor-client-plugins/ktor-client-auth/common/src/io/ktor/client/plugins/auth/providers/BearerAuthProvider.kt#L138-L143

https://github.com/ktorio/ktor/blob/5f5947ebae390cece2c1a22ac92d5651d764471d/ktor-client/ktor-client-plugins/ktor-client-auth/common/src/io/ktor/client/plugins/auth/providers/AuthTokenHolder.kt#L59-L69

A workaround for this is for ktor consumers to wrap their refreshTokens logic in a try block that catches Throwable like this:

install(Auth) {
  bearer {
    loadTokens {
      // ...
    }
    refreshTokens {
      try {
        // ...
      } catch (t: Throwable) {
        // don't allow this to bubble up
      }
    }
  }
}

Summary

Ktor 2.3.0 client under Javascript is unable to stream responses from a server. The same code works correctly with Ktor running under the JVM.

Steps to reproduce

1. Configure a server to emit responses as a stream
2. Configure a ktor client in Javascript to read the response as a stream (e.g. response.bodyAsChannel() and reading response line by line)

With the attached sample application

1. Run the server ./gradlew :server-app:run and make sure you have a console to look at the server log output
2. Launch your browser and open the developer tools
3. Visit http://localhost:8080
4. Compare the logs from the client and server

Results

Expected

Javascript client reads stream as each item is written

Actual

Javascript buffers until entire response is complete before actually reading it.

Example output from the test application
Server log output:

Writing H at 2023-04-29T16:56:47.240733Z
Writing e at 2023-04-29T16:56:47.750986Z
Writing l at 2023-04-29T16:56:48.256360Z
Writing l at 2023-04-29T16:56:48.761753Z
Writing o at 2023-04-29T16:56:49.264868Z
Writing   at 2023-04-29T16:56:49.765610Z
Writing w at 2023-04-29T16:56:50.270883Z
Writing o at 2023-04-29T16:56:50.776197Z
Writing r at 2023-04-29T16:56:51.283105Z
Writing l at 2023-04-29T16:56:51.788978Z
Writing d at 2023-04-29T16:56:52.292238Z

Browser log output:

Received H at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received e at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received l at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received l at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received o at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received w at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received o at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received r at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received l at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received d at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)

Notes

I reproduced this with Ktor 2.3.0. I have not tested older versions to see whether this is a long standing bug or a regression.

Expected behavior: Coroutine cancellation leads WebSocket to be closed by Normal Closure (1000) reason or like so

Actual behavior: Any coroutine completion with non-null cause leads WebSocket to be closed by Internal Error (1011) reason

Also, this reason for closing is not supported by browsers, which causes an invalid access error, leaving the WebSocket open.

For example, in HOCON I can write the following:

value {
  my = "My value"
}

config {
  database {
    value = ${value.my}
  }
}

If I write similar config file in YAML:

value:
  my: "My value"

config:
  database:
    value: $value.my

It will be failing to read $value.my assuming it's an environment variable

Steps to reproduce:

1. Create a Ktor application

import io.ktor.server.application.*
import io.ktor.server.cio.*
import io.ktor.server.routing.*

fun main(args: Array<String>) {
    EngineMain.main(args)
}

@Suppress("unused")
fun Application.main() {
    routing {
    }
}

2. Create two external config files

application.conf

ktor {
  development = false

  deployment {
    host = "0.0.0.0"
    port = 8080
    shutdownGracePeriod = 2000
    shutdownTimeout = 60000
  }

  application {
    modules = [package.MainKt.main]
  }
}

extra.conf

// Ktor can't load additional configuration without "ktor" definition
//ktor {
//  deployment {
//  }
//}

app {
  test = "conf"
}

3. Run the app using external configuration files
   -config=application.conf -config=extra.conf

The result

Actual
It fails with Exception in thread "main" com.typesafe.config.ConfigException$Missing: extra.conf: 1: No configuration setting found for key 'ktor'

Expected
The app should run since the ktor configuration is defined in the application.conf file

As a workaround, it works when empty ktor and deployment configurations are added to the extra.conf file.

KotlinxSerializationConverter#deserialize() “hides” actual deserialization error messages:

// KotlinxSerializationConverter.kt
fun deserialize(): Any? {
  try {
    // Deserialize and return if successful.
  } catch (cause: Throwable) {
    throw JsonConvertException("Illegal input", cause)
  }
}

While the message will be available in the cause of the thrown JsonConvertException , it causes the Logging plugin to display some pretty useless logs:

// Logging.kt
fun logResponseException(log: StringBuilder, request: HttpRequest, cause: Throwable) {
  log.append("RESPONSE ${request.url} failed with exception: $cause")
}

The result is logs like this, which unfortunately doesn't provide much value:

RESPONSE https://host/path failed with exception: io.ktor.serialization.JsonConvertException: Illegal input

Is there a way we can provide cause.message alongside the "Illegal input" message from KotlinxSerializationConverter? Perhaps just something like this:

throw JsonConvertException("Illegal input: ${cause.message}", cause)

Currently, upon receipt of an `X-Forwarded-For` header, the XForwardedHeaders plugin sets the remoteHost origin property:

https://github.com/ktorio/ktor/blob/e26794418af9bd2e0f38dc25cd938868ffa71fc5/ktor-server/ktor-server-plugins/ktor-server-forwarded-header/jvmAndNix/src/io/ktor/server/plugins/forwardedheaders/XForwardedHeaders.kt#L172

If the value extracted from the header is an address rather than a host, Ktor should be setting the remoteAddress property.

The KDocs for these two properties are:

/** Client address or host name if it can be resolved. ... */
public val remoteHost: String

and

/** Client address. ... */
public val remoteAddress: String

which clearly implies that for an address, these two values should be the same. The kdocs for remoteAddress even specify that it can be overwridden via the forwarded headers.

Therefore, client code that wants the remote address should clearly be using remoteAddress, and yet when the XForwardedHeaders plugin is used and the headers resolve to an address, the remoteHost will contain the resolved remote address, but remoteAddress will not.

Hi, I'm new to Ktor.
please understand if my question is wrong.

when I use session like below

install(Sessions) {
        cookie<User>(
            USER_COOKIE_NAME,
            directorySessionStorage(File(".sessions"), cached = true)
        )

I set session on signin api is called.
and client saved the sessionId and delivered the sessionId to server on getUser api like below
and server response with Set-Cookie header even getUser api controller didn't set session.

REQUEST: http://192.168.224.95:8080/getUser
METHOD: HttpMethod(value=POST)
COMMON HEADERS
-> KEY: Header test
-> Accept: application/json
-> Accept-Charset: UTF-8
-> Cookie: user=4879fe5aee4c5b3d;
CONTENT HEADERS
BODY Content-Type: application/json
BODY Content-Type: application/json; charset=UTF-8
BODY START
RESPONSE: 200 OK
METHOD: HttpMethod(value=POST)
FROM: http://192.168.224.95:8080/getUser
COMMON HEADERS
-> Connection: keep-alive
-> Content-Length: 21
-> Content-Type: application/json; charset=UTF-8
-> Set-Cookie: simple-user=4879fe5aee4c5b3d; Max-Age=604800; Expires=Thu, 20 Aug 2020 15:57:08 GMT; Path=/; HttpOnly; $x-enc=URI_ENCODING
-> X-Android-Received-Millis: 1597334141402
-> X-Android-Response-Source: NETWORK 200
-> X-Android-Selected-Protocol: http/1.1
-> X-Android-Sent-Millis: 1597334137063
{"id":"a","name":"b"}
BODY END

so, I checked which part add Set-Session
and found this code on io.ktor.sessions.Sessions

pipeline.intercept(ApplicationCallPipeline.Features) {
                val providerData = sessions.providers.associateBy({ it.name }) {
                    it.receiveSessionData(call)
                }
                val sessionData = SessionData(sessions, providerData)
                call.attributes.put(SessionKey, sessionData)
            }

in my past experience, Set-Session is added only when session is set.
But, my experience is not long, so, kindly understand if I'm wrong.
I would like to ask if this is intended behavior.

Thanks to read.
Please let me know if further information is required.

I have Ktor server in an Android app and I use it to receive files. I would like to display a progress bar in my app for the duration it will take for the receiving file to arrive.

Some sort of a listener with bytesSent/bytesTotal would be perfect

Just to clarify, I am not interested in listening to upload progress from a client, but the time it takes to receive a files from the server.

Right now my incoming files handling code looks like this:

// 1
call.receiveMultipart().forEachPart { part ->
    // 2
    when (part) {
        is PartData.FileItem -> {
            val input = part.provider()
            // Read from input by chunks
        }
        else -> {}
    }
}

I would like to listen to the progress between (1) and (2). Right now (1) is called, then the file is fully uploaded to the server, and then (2) is called.

I am attempting to migrate an existing netty based application to ktor.
One of its features is handling large file uploads (100mb -> 10Gb).
We are handle sensitive personal information and would prefer not write anything to disk.
I can't see a way to implement this using the existing multipart file upload support in Ktor.

Is there a way to directly access a ReadByteChannel for the file and avoid the tmp disk write?

Platform: JVM

We recently observed our production application servers locking up and becoming unresponsive due to legacy clients making a large number of web socket upgrade requests to a URI for which a previous route no longer existed. This is equivalent to a denial-of-service attack. We traced the issue back to the simple reproduction below, which also exposes a Ktor web socket client ambiguity on how to handle 404 and other connection issues: the documentation is not clear about what happens in those cases. If catching an exception is the way to go, as in the reproduction below, please update the documentation.

The issue appears to be that the server pipeline for web socket upgrade requests never completes if the client's upgrade request is sent to a URI for which no web socket handler route exists. The attached reproduction starts a server and then attempts to make 1,000 web socket connections. As is, this succeeds and at the end of the test run, roughly 10 coroutines exist on the server.

If the configured route is changed from /test/{id} to /test, the server locks up after processing ~200 requests, meaning it doesn't become unresponsive immediately, but only past a certain load. Over 4,000 suspended coroutines exist on the server at the end of the failed test run.

The workaround we're testing right now is to accept and immediately drop the connections to the known legacy route, but that doesn't resolve the issue that a potential attacker who knows that Ktor with web socket support is running on our servers could craft a large number of web socket requests to a different non-existing route and cause application servers to become unresponsive!

In HOCON there are 3 scenarios of variable declaration:

a) With default value

myValue = "default value"
myValue = ${?MY_ENV_VAR}

myValue equals "default value" if one doesn't provide MY_ENV_VAR

b) Without default value and with question mark

myValue = ${?MY_ENV_VAR}

myValue equals the value of MY_ENV_VAR or does not exist otherwise (.getProperty returns null)

c) Without default value and with no question mark

myValue = $MY_ENV_VAR

myValue equals the value of MY_ENV_VAR which must be present (.getProperty throws an exception in attempt to read MY_ENV_VAR)

There are only 2 equivalents in YAML:
a) With default value

myValue = "$MY_ENV_VAR:default value"

myValue equals "default value" if one doesn't provide MY_ENV_VAR

c) With no default value

myValue = $MY_ENV_VAR

myValue equals the value of MY_ENV_VAR which must be present (.getProperty throws an exception in attempt to read MY_ENV_VAR)

The only way to achieve "b" in YAML is to write ugly code like the following introducing an extra "NULL" string (Or any other token that one believes will never present in real values):

my:
  config: "$MY_ENV_VAR:NULL"

And then handling it like the following.

fun ApplicationConfig.yamlStringPropertyOrNull(path: String): String? = propertyOrNull(path)
    ?.getString()
    ?.takeIf { it != "NULL" }

fun ApplicationConfig.yamlListProperty(path: String): List<String> = property(path)
    .getList()
    .filter { it != "NULL" }

val myValue get(): MyConfig = conf.yamlStringPropertyOrNull("my.config")

When using the example here I get a java.lang.NullPointerException when running the test. Am I doing something wrong? My hope is to be able to test my WebSocket client of my KMP-app and I got the impression that this might be able to help me?

Using Ktor 2.2.4 and Kotlin 1.8.10.

Here's the stacktrace of the crash. Thanks in advance.

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

java.lang.NullPointerException
	at io.ktor.server.testing.TestApplicationEngineJvmKt$handleWebSocketConversationNonBlocking$5.invokeSuspend(TestApplicationEngineJvm.kt:73)
	(Coroutine boundary)
	at io.ktor.server.testing.TestApplicationEngineJvmKt.handleWebSocketConversationNonBlocking(TestApplicationEngineJvm.kt:68)
	at io.ktor.server.testing.client.TestHttpClientEngineBridge.runWebSocketRequest(TestHttpClientEngineBridgeJvm.kt:34)
	at io.ktor.server.testing.client.TestHttpClientEngine.execute(TestHttpClientEngine.kt:50)
	at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:99)
	at io.ktor.client.engine.HttpClientEngine$DefaultImpls.executeWithinCallContext(HttpClientEngine.kt:100)
	at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:70)
	at io.ktor.client.plugins.HttpSend$DefaultSender.execute(HttpSend.kt:138)
	at io.ktor.client.plugins.HttpRedirect$Plugin$install$1.invokeSuspend(HttpRedirect.kt:64)
	at io.ktor.client.plugins.HttpCallValidator$Companion$install$3.invokeSuspend(HttpCallValidator.kt:151)
	at io.ktor.client.plugins.HttpSend$Plugin$install$1.invokeSuspend(HttpSend.kt:104)
	at io.ktor.client.plugins.websocket.WebSockets$Plugin$install$1.invokeSuspend(WebSockets.kt:169)
	at io.ktor.client.plugins.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:130)
	at io.ktor.client.plugins.HttpRequestLifecycle$Plugin$install$1.invokeSuspend(HttpRequestLifecycle.kt:38)
	at io.ktor.client.HttpClient.execute$ktor_client_core(HttpClient.kt:191)
	at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:108)
	at io.ktor.client.plugins.websocket.BuildersKt.webSocket(builders.kt:241)
	at ModuleTest$testConversation$1.invokeSuspend(ModuleTest.kt:15)
	at io.ktor.server.testing.TestApplicationKt$testApplication$builder$1$1.invokeSuspend(TestApplication.kt:288)
Caused by: java.lang.NullPointerException
	at io.ktor.server.testing.TestApplicationEngineJvmKt$handleWebSocketConversationNonBlocking$5.invokeSuspend(TestApplicationEngineJvm.kt:73)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)

The Rate limit provider with name RateLimitName (name=KTOR_NO_NAME_RATE_LIMITER) is not configured or a similar error doesn't explain that the RateLimit plugin must be installed if it wasn't and that its installation must go before the one of the routing.

In case the plugin is set up to have separate limiter per user, this can cause unnecessary memory usage.

https://github.com/ktorio/ktor/blob/1d110bc08f0bdebb96d5370dd1a94ea65e64ec2e/ktor-shared/ktor-websockets/common/src/io/ktor/websocket/DefaultWebSocketSession.kt#L78

        val calendar = Calendar.getInstance(TimeZone.getTimeZone("Europe/Berlin"))
        val timestamp = calendar.toDate(null).timestamp // Equals to GMTDate().timestamp

After creating a new GMTDate instance from this timestamp, a different date is created.

In the engine there are several places which check first if (PlatformUtils.IS_BROWSER) but it would be better if those checked for if (PlatformUtils.IS_NODE) first (i.e. flip the if and else blocks) because for Electron IS_NODE and IS_BROWSER can both be true at the same time, but the browser fetch API is very crippled in comparison to the node-fetch based solution. For example you can't read response headers without explicitly setting up CORS headers in the backend.

In general, instead of having two Booleans (which can both be true) it would be better to have an enum and a single place in the code where the decision of prioritizing Node vs. browser behavior happens. Otherwise such "bugs" of prioritizing crippled browser APIs will happen all over the place and harm Electron integration.

When the server sends a PING frame, the Ktor client replies with 2 PONG frames.

Maybe the Java web socket client implementation already deals with PING/PONG automatically, and Ktor adds another layer of PING handling, resulting in 2 PONGs, but I haven't checked the code.

It is technically valid to send unsolicited PONG frames as per the web socket specification, but this is not quite what I would expect, and it's breaking Autobahn TestSuite, marking Ktor as failing ping/pong tests.

I'm experiencing that the DarwinClientEngine deadlocks on any http requests since 2.2.2 if they are started from the main thread. If I downgrade to 2.2.1 everything works fine.

deadlocks:

HttpClient(
    Darwin.create {
        pipelining = true
    }
)

println("making request..")
val result = httpClient().get("https://www.google.com")
println("status: $result.status")

works:

HttpClient(
    Darwin.create {
        pipelining = true
    }
)

launch(Dispatchers.Default) {
    println("making request..")
    val result = httpClient().get("https://www.google.com")
    println("status: $result.status")
}

File Websockets.kt contains own LOGGER

internal val LOGGER = KtorSimpleLogger("io.ktor.client.plugins.websocket.WebSockets")

Which can't be configured.

Implementation of KtorSimpleLoggerJs.kt

@Suppress("FunctionName")
public actual fun KtorSimpleLogger(name: String): Logger = object : Logger {
    override fun error(message: String) {
        console.error(message)
    }

    override fun error(message: String, cause: Throwable) {
        console.error("$message, cause: $cause")
    }

    override fun warn(message: String) {
        console.warn(message)
    }

    override fun warn(message: String, cause: Throwable) {
        console.warn("$message, cause: $cause")
    }

    override fun info(message: String) {
        console.info(message)
    }

    override fun info(message: String, cause: Throwable) {
        console.info("$message, cause: $cause")
    }

    override fun debug(message: String) {
        console.info("DEBUG: $message")
    }

    override fun debug(message: String, cause: Throwable) {
        console.info("DEBUG: $message, cause: $cause")
    }

    override fun trace(message: String) {
        console.info("TRACE: $message")
    }

    override fun trace(message: String, cause: Throwable) {
        console.info("TRACE: $message, cause: $cause")
    }
}

It floods logs with unnecessary and useless messages (why not console.trace?):

TRACE: Sending WebSocket request [object Object]

This logger should be configurable with Logging plugin.

We are using BearerAuthProvider and experiencing unexpected consecutive refresh token calls in this particular scenario.

Here the sequence of the calls with url, response code, response time and token. The calls are in response order.

1. /url1 -> 401 - response time: 310 ms - uses token1
2. /url2 -> 401 - response time: 276 ms - uses token1
3. /url3 -> 401 - response time: 322 ms - uses token1
4. /refresh -> 200 - response time: 390 ms - refresh token 1 to token 2
5. /url1 -> 200 - response time: 168 ms - uses token 2
6. /url2 -> 200 - response time: 179 ms - uses token 2
7. /url3 -> 200 - response time: 189 ms - uses token 2
8. /url4 -> 401 - response time: 264 ms - uses token 1
9. /refresh -> 200 - response time: 168 ms - refresh token 2 to token 3
   ...

What we see is that:

• url1, url2, url3 are queued for retry and retried after step 4 in which the refresh token is done. These retries succeed.
• url4 is started when the refresh token is in process but the 401 is received after the refresh token finishes. The 401 failure of url4 leads to another refresh token. We were expecting that this call was going to be retried immediately without a refresh token, just using token2 (refreshed token received at step 4).

The situation gets even worse if we amplify this behaviour to a higher number of calls: e.g. this could lead to url4 failing twice with 401 and this call will be lost and never complete.

realm should be set earlier in the function

https://github.com/ktorio/ktor/blob/ebaf9f16786566a3061ebd4fc9de5523c0291025/ktor-client/ktor-client-plugins/ktor-client-auth/common/src/io/ktor/client/plugins/auth/providers/DigestAuthProvider.kt#L165

should be moved to before line 154. This fixes the issues for me.

Currently in JavaHttpWebSocket.onError, Ktor throws a WebSocketException with just the message from the cause:

    override fun onError(webSocket: WebSocket, error: Throwable) {
        val cause = WebSocketException("${error.message}")
        _incoming.close(cause)
        _outgoing.cancel()
        socketJob.complete()
    }

https://github.com/ktorio/ktor/blob/main/ktor-client/ktor-client-java/jvm/src/io/ktor/client/engine/java/JavaHttpWebSocket.kt#L186

This is not a nice solution in general, because exception messages don't completely inform about the problem. It's the combination of exception type + message that gives the full insight. For instance, with FileNotFoundException: /tmp/foo, we don't want to just bubble up the message /tmp/foo, it doesn't really explain the problem.

As a more realistic example, I'm getting a "null" message from Ktor because some exception in the Java web socket has no message, which makes this stacktrace pretty pointless:

Caused by: org.hildan.krossbow.websocket.WebSocketException: error in Ktor's websocket: null	
at org.hildan.krossbow.websocket.ktor.KtorWebSocketConnectionAdapter$incomingFrames$3.invokeSuspend(KtorWebSocketClient.kt:60)	
•••
at org.hildan.krossbow.websocket.test.TestUtilsJvmKt.runSuspendingTest(TestUtilsJvm.kt:8)	
at org.hildan.krossbow.websocket.test.TestUtilsJvmKt.runSuspendingTest$default(TestUtilsJvm.kt:8)	
at org.hildan.krossbow.websocket.test.autobahn.AutobahnClientTestSuite.runAutobahnTestCase(AutobahnClientTestSuite.kt:200)	
at org.hildan.krossbow.websocket.test.autobahn.AutobahnClientTestSuite.autobahn_1_1_7_echo_text_payload(AutobahnClientTestSuite.kt:54)	
•••
Caused by: io.ktor.client.plugins.websocket.WebSocketException: null	
at (Coroutine boundary.()	
•••
Caused by: io.ktor.client.plugins.websocket.WebSocketException: null	
at io.ktor.client.engine.java.JavaHttpWebSocket.onError(JavaHttpWebSocket.kt:186)	

And the chain of causes stops there. See this build scan for a full example stack trace.

Could you please add the cause here to the WebSocketException?

I've stumbled upon improper code generated from swagger spec, roughly equivalent to:

append("key[]", listOf("value0", "value1", "etc"))

Although it is easy to convert to something like:

listOf("value0", "value1", "etc").forEach { append("key[]", it) }

Seems presence of catch-all append<T> is confusing users:

    /**
     * Appends a pair [key]:[value] with optional [headers].
     */
    @InternalAPI
    public fun <T : Any> append(key: String, value: T, headers: Headers = Headers.Empty) {
        parts += FormPart(key, value, headers)
    }

I propose adding additional append overload to support List<String> and optionally List<Number> parameters:

    /**
     * Appends a pair [key]:[values] with optional [headers].
     */
    public fun append(key: String, values: List<String>, headers: Headers = Headers.Empty) {
        require(key.endsWith("[]")) {
            "Array parameter must be suffixed with square brackets ie `key[]`"
        }
        values.forEach { value ->
            parts += FormPart(key, value, headers)
        }
    }
    

We are building a Kotlin Native project for desktop. We decided to use ktor-client-curl, as it supports both Mac and Windows. On the other hand, currently the client fails on Windows with the following exception when using any HTTPS URLs:

io.ktor.client.engine.curl.CurlIllegalStateException: TLS verification failed for request: CurlRequestData(url='https://google.com', method='GET', content: 0 bytes). Reason: SSL peer certificate or SSH remote key was not OK
io.ktor.client.engine.curl.CurlIllegalStateException: TLS verification failed for request: CurlRequestData(url='https://google.com', method='GET', content: 0 bytes). Reason: SSL peer certificate or SSH remote key was not OK
	at kotlin.Throwable#<init>(Unknown Source)
	at kotlin.Exception#<init>(Unknown Source)
	at kotlin.RuntimeException#<init>(Unknown Source)
	at kotlin.IllegalStateException#<init>(Unknown Source)
	at io.ktor.client.engine.curl.CurlIllegalStateException#<init>(Unknown Source)
	at io.ktor.client.engine.curl.internal.CurlMultiApiHandler.collectFailedResponse#internal(Unknown Source)
	at io.ktor.client.engine.curl.internal.CurlMultiApiHandler.processCompletedEasyHandle#internal(Unknown Source)
	at io.ktor.client.engine.curl.internal.CurlMultiApiHandler.handleCompleted#internal(Unknown Source)
	at io.ktor.client.engine.curl.internal.CurlMultiApiHandler#perform(Unknown Source)
	at io.ktor.client.engine.curl.CurlProcessor.$runEventLoop$lambda$3COROUTINE$17.invokeSuspend#internal(Unknown Source)
	at kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(Unknown Source)
	at kotlinx.coroutines.DispatchedTask#run(Unknown Source)
	at io.ktor.util.MultiWorkerDispatcher.$workerRunLoop$lambda$4COROUTINE$80.invokeSuspend#internal(Unknown Source)
	at kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(Unknown Source)
	at kotlinx.coroutines.DispatchedTask#run(Unknown Source)
	at kotlinx.coroutines.EventLoopImplBase#processNextEvent(Unknown Source)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal(Unknown Source)
	at kotlinx.coroutines#runBlocking(Unknown Source)

It looks like it's not using the certificate store of Windows, and for me it also failed to detect the ca-cert.pem file regardless where I copied it (see https://github.com/ktorio/ktor/issues/671#issuecomment-619832512 for suggestions).

It would be great if the CURLOPT_CAINFO and CURLOPT_CAPATH options would be configurable for the Ktor Curl client as this way I could directly provide the path to the ca-cert.pem file.

Description
After updating from KTOR 2.1.2 to 2.2.2 and adding HttpCache I started to receive bug reports from my testers regarding a GZIP magic invalid: 12345 error. After some testing I realized that only release builds but not debug builds were affected and traced the issue to the LogLevel used, which is LogLevel.INFO for release and LogLevel.ALL for debug. Also LogLevel.HEADERS is affected by this issue.

Updating to KTOR 2.2.3 did not bring any improvements.

Setup

• KTOR 2.2.2 or 2.2.3
• OkHttp Engine on Android is broken
• Darwin Engine on iOS works as expected

The following configuration triggers the issue. It happens with and without setting the public/private storage set to my storage implementation. To ensure my storage doesn't cause the issue, I commented it out with the same result. Setting one or the other or both makes no difference.

To fix the problem, one of three things needs to happen:

• Remove the install(HttpCache) , commenting out the storages as shown does not help
• Remove the install(ContentEncoding), maybe removing GZIP is sufficient, but the server doesn't provide Deflate so I can't test this
• Set level to LogLevel.ALL

install(HttpCache) {
//  publicStorage(storage) 
//  privateStorage(storage)
}

install(ContentEncoding) {
    deflate(1.0f)
    gzip(0.9f)
}

install(Logging) {
    level = LogLevel.INFO
    logger = object  : Logger {
        override fun log(message: String) {
            println(message)
        }
    }
}

Stack trace

java.lang.IllegalStateException: GZIP magic invalid: 958
	at java.lang.reflect.Constructor.newInstance0(Native Method)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:343)
	at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$1.invoke(ExceptionUtilsJvm.kt:3)
	at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$1.invoke(ExceptionUtilsJvm.kt:1)
	at io.ktor.utils.io.ExceptionUtilsJvmKt.tryCopyException(ExceptionUtilsJvm.kt:77)
	at io.ktor.utils.io.ByteBufferChannelKt.rethrowClosed(ByteBufferChannel.kt:1)
	at io.ktor.utils.io.ByteBufferChannelKt.access$rethrowClosed(ByteBufferChannel.kt:1)
	at io.ktor.utils.io.ByteBufferChannel.setupStateForRead(ByteBufferChannel.kt:39)
	at io.ktor.utils.io.ByteBufferChannel.readAsMuchAsPossible(ByteBufferChannel.kt:34)
	at io.ktor.utils.io.ByteBufferChannel.readAvailable$suspendImpl(ByteBufferChannel.kt:1)
	at io.ktor.utils.io.ByteBufferChannel.readAvailable(ByteBufferChannel.kt:3)
	at io.ktor.utils.io.jvm.javaio.InputAdapter$loop$1.loop(Blocking.kt:126)
	at io.ktor.utils.io.jvm.javaio.InputAdapter$loop$1$loop$1.invokeSuspend(Blocking.kt:12)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:12)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:119)
	at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:11)
	at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:28)
	at io.ktor.utils.io.jvm.javaio.BlockingAdapter.submitAndAwait(Blocking.kt:16)
	at io.ktor.utils.io.jvm.javaio.BlockingAdapter.submitAndAwait(Blocking.kt:3)
	at io.ktor.utils.io.jvm.javaio.InputAdapter.read(Blocking.kt:6)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:248)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:267)
	// my app stack
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:12)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:119)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:13)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:3)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:1)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:15)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:29)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:1)
Caused by: java.lang.IllegalStateException: GZIP magic invalid: 958
	at io.ktor.util.EncodersJvmKt$inflate$1.invokeSuspend(EncodersJvm.kt:666)
	at io.ktor.util.EncodersJvmKt$inflate$1.invoke(EncodersJvm.kt:1)
	at io.ktor.util.EncodersJvmKt$inflate$1.invoke(EncodersJvm.kt:2)
	at io.ktor.utis.io.CoroutinesKt$launchChannel$job$1.invokeSuspend(Coroutines.kt:68)
	... 8 more

While playing around with logger settings I also got this error. It seems like the file I download determines which of the two exceptions I get. Downloading the same file multiple times consistently reproduces the same exception, across app restarts.

java.lang.IllegalArgumentException: max shouldn't be negative: -22207
    at java.lang.reflect.Constructor.newInstance0(Native Method)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:343)
    at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$1.invoke(ExceptionUtilsJvm.kt:103)
    at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$1.invoke(ExceptionUtilsJvm.kt:90)
    at io.ktor.utils.io.ExceptionUtilsJvmKt.tryCopyException(ExceptionUtilsJvm.kt:66)
    at io.ktor.utils.io.ByteBufferChannelKt.rethrowClosed(ByteBufferChannel.kt:2400)
    at io.ktor.utils.io.ByteBufferChannelKt.access$rethrowClosed(ByteBufferChannel.kt:1)
    at io.ktor.utils.io.ByteBufferChannel.setupStateForRead(ByteBufferChannel.kt:295)
    at io.ktor.utils.io.ByteBufferChannel.readAsMuchAsPossible(ByteBufferChannel.kt:2459)
    at io.ktor.utils.io.ByteBufferChannel.readAvailable$suspendImpl(ByteBufferChannel.kt:672)
    at io.ktor.utils.io.ByteBufferChannel.readAvailable(Unknown Source:0)
    at io.ktor.utils.io.jvm.javaio.InputAdapter$loop$1.loop(Blocking.kt:38)
    at io.ktor.utils.io.jvm.javaio.InputAdapter$loop$1$loop$1.invokeSuspend(Unknown Source:14)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
    at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:313)
    at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:201)
    at io.ktor.utils.io.jvm.javaio.BlockingAdapter.submitAndAwait(Blocking.kt:231)
    at io.ktor.utils.io.jvm.javaio.BlockingAdapter.submitAndAwait(Blocking.kt:201)
    at io.ktor.utils.io.jvm.javaio.InputAdapter.read(Blocking.kt:65)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:248)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:267)
    // my app stack
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
    at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
    at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
 Caused by: java.lang.IllegalArgumentException: max shouldn't be negative: -22207
    at io.ktor.utils.io.ByteBufferChannel.discard$suspendImpl(ByteBufferChannel.kt:1667)
    at io.ktor.utils.io.ByteBufferChannel.discard(Unknown Source:0)
    at io.ktor.util.EncodersJvmKt$inflate$1.invokeSuspend(EncodersJvm.kt:163)
    at io.ktor.util.EncodersJvmKt$inflate$1.invoke(Unknown Source:8)
    at io.ktor.util.EncodersJvmKt$inflate$1.invoke(Unknown Source:4)
    at io.ktor.utils.io.CoroutinesKt$launchChannel$job$1.invokeSuspend(Coroutines.kt:134)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) 
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) 
    at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42) 
    at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95) 
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570) 
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) 
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677) 
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664) 

Response of the server

Following file is downloaded
CE3PRO_pikachu_1gen_flowalistik.gcode

{width=70%}

Extracted this comment to a dedicated issue: https://youtrack.jetbrains.com/issue/KTOR-481/Ktor-client-logging-Add-an-option-to-filter-logged-calls#focus=Comments-27-6332247.0-0

I would like to have the ability to log debug headers like request-id trace headers while not logging sensitive Bearer token headers

I'm trying to use Ktor to talk to the REST Api of Shelly Smart Home devices, which use an lwIP webserver.

With a simple client set up like follows:

val httpClient = HttpClient(CIO)

any simple GET request fails with IllegalStateException:

Failed to parse request body: request body length should be specified,
chunked transfer encoding should be used or
keep-alive should be disabled (connection: close)

because the server neither sends Content-Length, nor Connection headers. In fact it doesn't send anything but Content-Type and Server :

HTTP/1.0 200 OK
Server: lwIP/2.1.2 (http://savannah.nongnu.org/projects/lwip)
Content-Type: application/json

I understand that this service should probably "behave better", but that's out of my control unfortunately. I debugged into io.ktor.http.cio.HttpBody.parseHttpBody and it seems to be that the issue would be solvable by defaulting to Connection: close if no Connection Header is given; and it seems this is (at least according to wikipedia) in HTTP/1.0 what the default should be; only in HTTP/1.1 connections default to being persistent.

Thanks!

Currently Boolean is not supported as a value for form data. However it could be treated the same way as Number is treated.

https://github.com/ktorio/ktor/blob/main/ktor-client/ktor-client-core/common/src/io/ktor/client/request/forms/formDsl.kt#L63

Ktor 2.1.3

package transport.ktor

import io.ktor.network.selector.*
import io.ktor.network.sockets.*
import kotlinx.coroutines.*
import kotlin.coroutines.*
import kotlin.random.*
import kotlin.test.*

class KtorReadingFromClosedSocketBugTest {
    @Test
    fun `Native - Read from a closed socket doesn't throw an exception`() {
        val address = InetSocketAddress("localhost", Random.nextInt(2_000..65_000))
        val tcp = aSocket(SelectorManager(EmptyCoroutineContext)).tcp()
        val serverSocket = tcp.bind(address)
        runBlocking {
            launch {
                serverSocket.accept()
                println("client connected")
            }
            val clientSocket = tcp.connect(address)
            launch {
                val clientReadChannel = clientSocket.openReadChannel()
                println("clientSocket.isClosed: ${clientSocket.isClosed}")
                println("clientReadChannel.isClosedForRead: ${clientReadChannel.isClosedForRead}")
                delay(2_000)
                println("clientSocket.isClosed: ${clientSocket.isClosed}")
                println("clientReadChannel.isClosedForRead: ${clientReadChannel.isClosedForRead}")
                try {
                    println("trying to read from clientReadChannel")
                    clientReadChannel.readByte()
                    println("reading from clientReadChannel succeeded^")
                } catch (e: Exception) {
                    println("reading from clientReadChannel failed")
                }
            }
            delay(1_000)
            clientSocket.close()
            println("clientSocket closed")
        }
        /*
            jvm target (expected behavior)              native target (erroneous behavior)
            ------------------------------              ----------------------------------
            client connected                            client connected
            clientSocket.isClosed: false                clientSocket.isClosed: false
            clientReadChannel.isClosedForRead: false    clientReadChannel.isClosedForRead: false
            clientSocket closed                         clientSocket closed
            clientSocket.isClosed: true                 clientSocket.isClosed: false
            clientReadChannel.isClosedForRead: true     clientReadChannel.isClosedForRead: false
            trying to read from clientReadChannel       trying to read from clientReadChannel
            reading from clientReadChannel failed       <reading from clientReadChannel never returns>
        */
    }
}

Please see Github for code.

in 1.9.0 K/JS legacy compiler is deprecated, so Ktor is broken in user projects:

e: Old Kotlin/JS compiler is no longer supported. Please migrate to the new JS IR backend

https://buildserver.labs.intellij.net/buildConfiguration/Kotlin_KotlinDev_ktor/265268196?hideProblemsFromDependencies=false&hideTestsFromDependencies=false&expandBuildProblemsSection=true&showLog=265268196_5643_683.5698&logFilter=debug&logView=flowAware

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1091

Ktor Version

ktor-http 1.1.5 and below

Feedback

Add a parameters { ... } builder DSL to replace the less discoverable Parameters.build { ... } companion object function in favor of being more consistent with the stdlib.

sequenceOf(1, 2, 3)/listOf(1, 2, 3) -> parametersOf("first", "a")
sequence { ... } -> parameters { ... }

call.respondRedirect("https://www.google.com")

Note: in Ktor, this function is implemented as following:

public suspend fun ApplicationCall.respondRedirect(url: String, permanent: Boolean = false) {
    response.headers.append(HttpHeaders.Location, url)
    respond(if (permanent) HttpStatusCode.MovedPermanently else HttpStatusCode.Found)
}

Expected: if permanent = false, HttpStatusCode.MovedPermanently should be the status code, and HttpStatusCode.Found otherwise.
Actual: no information in OpenAPI file

While it works fine in main module, if you add a separate YAML config for tests, that's what you'll see:

intall(StatusPages) {
  exception<E> { ... }
  unhandledException()
}

fun StatusPagesConfig.unhandledException() {
  call.respond(...)
}

Here, call.respond from unhandledException must be present in OpenAPI as well.

Example:

get("/a/b/c") {
  call.respond(randomCoordinate())
  exampleRespond(Coordinate(10, 20))
}

class Coordinate(val x: Int, val y: Int)

fun randomCoordinate(): Coordinate = Coordinate(Random.nextInt(), Random.nextInt())

fun Route.exampleRespond(value: Any?) {
  // A hack to add OpenAPI examples without affection actuall HTTP behaviour:
  if (false) { 
    call.respond(value)
  }
}

Expected: OpenAPI shows that /a/b/c route has #ref/Coordinate response type reference.
(Ideally, it also has to show value: x: 10 y: 20 as an example value for that response type, but it is not supported yet (KTOR-5683))
Actual: OpenAPI shows that /a/b/c route has an object response type.

To reproduce:
Open Qodana Cloud and try to generate OpenAPI for core-api module.
Exception:

java.lang.StackOverflowError
	at org.jetbrains.kotlin.idea.caches.resolve.ResolutionFacadeWithDebugInfo.analyze(ResolutionFacadeWithDebugInfo.kt:43)
	at org.jetbrains.kotlin.idea.caches.resolve.ExtendedResolutionApiKt.analyze(ExtendedResolutionApi.kt:124)
	at org.jetbrains.kotlin.idea.caches.resolve.ResolutionUtils.analyze(ResolutionUtils.kt:123)
	at org.jetbrains.uast.kotlin.internal.IdeaKotlinUastResolveProviderService.getBindingContext(IdeaKotlinUastResolveProviderService.kt:25)
	at org.jetbrains.uast.kotlin.internal.IdeaKotlinUastResolveProviderService.getBindingContextIfAny(IdeaKotlinUastResolveProviderService.kt:28)
	at org.jetbrains.uast.kotlin.KotlinInternalUastUtilsKt.analyze(kotlinInternalUastUtils.kt:232)
	at org.jetbrains.uast.kotlin.KotlinUastResolveProviderService.getExpressionType(KotlinUastResolveProviderService.kt:309)
	at org.jetbrains.uast.kotlin.KotlinUastResolveProviderService.getType(KotlinUastResolveProviderService.kt:304)
	at org.jetbrains.uast.kotlin.KotlinUElementWithType.getExpressionType(KotlinUElementWithType.kt:16)
	at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.getExpressionType(KotlinUFunctionCallExpression.kt:92)
	at io.ktor.ide.oas.KtorOasStatusCodeVisitor.visitCallExpression(KtorOasArgumentVisitors.kt:51)
	at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:165)
	at org.jetbrains.uast.kotlin.KotlinUField.accept(KotlinUField.kt:59)
	at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
	at org.jetbrains.uast.kotlin.AbstractKotlinUClass.accept(AbstractKotlinUClass.kt:99)
	at io.ktor.ide.oas.KtorOasStatusCodeVisitor.visitExpression(KtorOasArgumentVisitors.kt:45)
	at org.jetbrains.uast.visitor.UastVisitor.visitSimpleNameReferenceExpression(UastVisitor.kt:27)
	at org.jetbrains.uast.USimpleNameReferenceExpression.accept(USimpleNameReferenceExpression.kt:19)
	at org.jetbrains.uast.UEnumConstant.accept(UVariable.kt:148)
	at io.ktor.ide.oas.KtorOasStatusCodeVisitor.visitExpression(KtorOasArgumentVisitors.kt:45)
	at org.jetbrains.uast.visitor.UastVisitor.visitSimpleNameReferenceExpression(UastVisitor.kt:27)
	at org.jetbrains.uast.kotlin.KotlinUSimpleReferenceExpression.accept(KotlinUSimpleReferenceExpression.kt:40)
	at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
	at org.jetbrains.uast.kotlin.KotlinUFunctionCallExpression.accept(KotlinUFunctionCallExpression.kt:169)
	at org.jetbrains.uast.kotlin.KotlinUField.accept(KotlinUField.kt:59)
	at org.jetbrains.uast.internal.ImplementationUtilsKt.acceptList(implementationUtils.kt:14)
	at org.jetbrains.uast.kotlin.AbstractKotlinUClass.accept(AbstractKotlinUClass.kt:99)
	at io.ktor.ide.oas.KtorOasStatusCodeVisitor.visitExpression(KtorOasArgumentVisitors.kt:45)
  ...

Example: Open Qodana Cloud project and generate OpenAPI for core-api module.
Computation will run for around 1 minute without an option to cancel.

Example:

get("/a/b/c") {
  call.respond(Coordinate(10, 20))
}

class Coordinate(val x: Int, val y: Int)

Expected: OpenAPI has to show value: x: 10 y: 20 as an example value for that response type of /a/b/c request.
Actual: example it is not reflected in OpenAPI.

Example:

call.respond(MyData(x = 1, y = 2, items = listOf("a", "b", "c")))

Or even:

call.respond(MyData(x = 1, y = 2))
...
class MyData(val x: Int, val y: Int, val items: List<String> = listOf("a", "b", "c"))

OpenAPI should have an example entrance:

examples:
  Example#1:
    description: ""
    value:
      x: 1
      y: 1
      items:
        - "a"
        - "b"
        - "c"

Example:

routing {
    get("/my/route") {
        sendValue(MyObject(1, 2))
    }
}
...
fun  <T>  PipelineContext<Unit, ApplicationCall>.sendValue(value: T) {
    call.respond(HttpStatusCode.Accepted, value)
}

Expected: After generating OpenAPI documentation, it should tell about MyObject(1, 2) response with example value based on what was passed to sendValue function.

Example:

data class PaginatedList<T>(
    @Expose val count: Long,
    @Expose val next: Long?,
    @Expose val prev: Long?,
    @Expose val items: List<T>,
)

data class ProjectListResponseItem(
    @Expose val id: String,
    @Expose val name: String,
    @Expose val slug: String = name.asSlug,
    @Expose val branch: String? = null,
    @Expose val commit: String? = null,
    @Expose val lastChecked: Instant? = null,
    @Expose @NoSlug val lastReportId: String? = null,
    @Expose val lastState: ReportState?,
    @Expose val url: String? = null,
    @Expose val problems: Problems? = null,
    @Expose val baselineCount: Int? = null,
)
...
call.respond(results())
...
fun results(): PaginatedList<ProjectListResponseItem>

Expected:

responses:
        "200":
          description: ""
          content:
            '*/*':
              schema:
                $ref: "#/components/schemas/PaginatedList_ProjectListResponseItem"
...
schemas:
    PaginatedList_ProjectListResponseItem:
      type: "object"
      properties:
        count:
          type: "integer"
          format: "int64"
        next:
          type: "integer"
          format: "int64"
        prev:
          type: "integer"
          format: "int64"
        items:
          type: "array"
          items:
            $ref: "#/components/schemas/ProjectListResponseItem"
    ProjectListResponseItem:
      type: "object"
      properties:
        id:
          type: "string"
        name:
          type: "string"
        slug:
          type: "string"
        branch:
          type: "string"
        commit:
          type: "string"
        lastChecked:
          $ref: "#/components/schemas/Instant"
        lastReportId:
          type: "string"
        lastState:
          type: "string"
          enum:
          - "UPLOADING"
          - "UPLOADED"
          - "PROCESSED"
          - "FAILED_TO_PROCESS"
          - "PINNED"
          - "MARKED_FOR_DELETION"
          - "OVERWRITTEN"
        url:
          type: "string"
        problems:
          $ref: "#/components/schemas/Problems"
        baselineCount:
          type: "integer"
          format: "int32"

Example:
There is a StatusPages plugin that handles exceptions:

exception<MyExceptionType> { call, cause ->
  val descriptor = ApiDescr(cause.name, cause.message)
  call.respond(cause.statusCode, descriptor) // <--- HERE
}

And routes:

get("/first/route") {
   throw MyExceptionType1("first name", "Cool message!")
}

get("/second/route") {
   throw MyExceptionType2("2nd name", "Bad message :(")
}

And exceptions:

class MyExceptionType(val statusCode: Int, val type: String, val details: String)
class MyExceptionType1:(type: String, details: String): MyExceptionType(HttpStatusCode.BAD_REQUEST, type, details)
class MyExceptionType2:(type: String, details: String): MyExceptionType(HttpStatusCode.UNAUTHORIZED, type, details)

Expected OpenAPI:

/first/route:
    get:
      description: ""
      responses:
        "400":
          description: "Bad Request"
          content:
            '*/*':
              schema:
                $ref: "#/components/schemas/ApiDescr"
              examples:
                Example#1:
                  value:
                    type: "first name"
                    details: "Cool message!"
/second/route:
    get:
      description: ""
      responses:
        "401":
          description: "Unauthorized"
          content:
            '*/*':
              schema:
                $ref: "#/components/schemas/ApiDescr"
              examples:
                Example#1:
                  value:
                    type: "2nd name"
                    details: "Bad message :("
components:
  schemas:
    APIError:
      type: "object"
      properties:
        type:
          name: "string"
        details:
          message: "string"

Actual OpenAPI: nothing is known about response types at all.

Note: even if status codes are known statically (written in exception<...> handlers), the APIError messages can not be deduced from exceptions.

Example:

ktor:
  application:
    modules:
      - a.b.c.myModule1
      - a.b.c.myModule2

fun Application.myModule1() {
 // routes
}

fun Application.myModule2() {
 // status pages
}

Expected: In this case, all exceptions thrown from myModule1 must be handled by StatusPages defined in myModule2. And this must be represented in OpenAPI.
Actual: OpenAPI generator does not know that myModule2 is also included to the application configuration and hence it does not analyze StatusPages from there because it is not directly referenced.

OpenAPI generator has to check application configuration files to know which other modules it has to analyze.

Example:

fun Application.main() {
  routing {
    f()
  }
}

fun Route.f() {
  get("/f") {...}
  g()
}

fun Route.g() {
  route("/g") {
    f()
  }
}

Expected: at least routes /f and g/f are shown.
Actual: no routes are shown, StackOverflow is thrown.

Example:

package a.b.c

import io.ktor.server.Application

fun Application.myModule() { ... }

yaml:

ktor:
  application:
    modules:
      - a.b.c.myModule

Expected: a.b.c.myModule is a reference in yaml leading to Application.myModule().
Actual: it does not resolve.

Example:

1. Without default values:

ktor:
  deployment:
    port: $PORT

2. And with default values:

ktor:
  deployment:
    port: "$PORT:8080"

Expected highlighting:


{width=70%}

Actual: currently, the whole "$PORT:8080" is seen as one single string that is a candidate to be converted to Integer, because variables are not supported.

Searching for "Content Negotiation" both in the web and IntelliJ wizards gives the following results:

I suppose this is because ContentNegotiation doesn't contain a space (even though other features do).
I suggest:

• Renaming the feature to Content Negotiation
• Ignoring whitespace in the fuzzy search.

Documentation for the Endpoints tool window says that WebSocket endpoints are supported:

The Endpoints tool window provides an aggregated view of both client and server APIs used in your project for HTTP and WebSocket protocols.

It would be useful to have this capability for Ktor projects. Currently, only HTTP routes are displayed.

I was surprised that kotlinx.coroutines Flow<T> is currently not supported by ktor-serialization, Flow is not collected so nothing happens.

What steps will reproduce the issue?

This is a minimal reproducer

fun main(args: Array<String>) {
    embeddedServer(Netty, 8080) {
        install(ContentNegotiation) {
            json()
        }
        routing {
            get("/") {
                call.respond(
                    flow {
                        emit(1)
                        delay(1000)
                        emit(2)
                    }
                )
            }
        }
    }.start(wait = true)
}

What is expected
As coroutines are heavilly used and supported in Ktor, I think ktor-serialization should support kotlinx.coroutines Flow<T> natively, as it is the official asynchronous Stream on Kotlin coroutines world.
I don't think that the Flow<T> support should require an additional Plugin, but you can of course disagre with my opinion ;)

The minimal support (on multiplatform) can be to just collect the Flow<T> as a List<T>, and then serialize it.

Sequence<T> should be available for deserialization too.

Bonus 1:

On JVM (with kotlinx.serialization, Jackson and Gson) normal and Flow<T> serialization can be optimized, and asynchronously write in the OutpuStream (server mode) and read from InpuStream (client mode), or using java.io.Writer/java.io.Reader. Examples are available here in a kotlinx.serialization JsonLazySequenceTest.kt
I made a custom implementation to support Flow in Ktor server mode in this class on my kotysa-ktor-r2dbc-coroutines sample project -> by the way if you have a few minutes to read this sample's very short code, and tell me if I made a mistake thanks a lot ^^

Bonus 2:

Flow<T> could also be used to asynchronously serialize data as Server Sent Events and in Websocket in ktor-websocket-serialization.

The unofficial application/x-json-stream (MediaType.APPLICATION_JSON_STREAM) could be supported aswell, even if not supported by Browsers and non standard yet, this format can be useful for server-to-server JSON communication. See this micronaut topic

https://github.com/ktorio/ktor/issues/514

/**
 * Support pre-compressed files in the file system only (not just any classpath resource)
 * ...
public fun Route.preCompressed( 

It looks like preCompressed only works with files() at the moment, why ?

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/119

We have continue support in netty however it shouldn't be done like that: we have to respond continue (to allow client sending request body) only when handler tries to access request body (call.request.receive()) or at least once request handler has started (that is not exactly correct as user's code could respond 404)

To reproduce run the following test:

class WebsocketsTest {
    private val server = embeddedServer(CIO, host = "127.0.0.1", port = 0) {
        install(WebSockets)

        routing {
            webSocket("/") {
                for (frame in incoming) {
                    outgoing.send(frame)
                }
            }
        }
    }

    private val client = HttpClient(io.ktor.client.engine.cio.CIO) {
        install(io.ktor.client.plugins.websocket.WebSockets)
    }

    @Before
    fun setup() {
        server.start(wait = false)
    }

    @After
    fun dispose() {
        server.stop()
    }

    @Test
    fun test() = testSuspend {
        val session = createSession()
        session.outgoing.send(Frame.Ping(ByteArray(126)))
        val frame = session.incoming.receive()

        assertIs<Frame.Close>(frame)
        assertEquals(1002, frame.readReason()?.code)
    }

    private suspend fun createSession(): ClientWebSocketSession {
        val url = "ws://127.0.0.1:${server.resolvedConnectors().first().port}/"
        return client.webSocketRawSession { url(url) }
    }
}

To reproduce run the following test:

class WebsocketsTest {
    private val server = embeddedServer(CIO, host = "127.0.0.1", port = 0) {
        install(WebSockets)

        routing {
            webSocket("/") {
                for (frame in incoming) {
                    outgoing.send(frame)
                }
            }
        }
    }

    private val client = HttpClient(io.ktor.client.engine.cio.CIO) {
        install(io.ktor.client.plugins.websocket.WebSockets)
    }

    @Before
    fun setup() {
        server.start(wait = false)
    }

    @After
    fun dispose() {
        server.stop()
    }

    @Test
    fun test() = testSuspend {
        val session = createSession()
        session.outgoing.send(Frame.Text(false, "hello ".toByteArray()))
        session.outgoing.send(Frame.Text(true, "world".toByteArray()))
        
        val frame = session.incoming.receive()
        assertIs<Frame.Close>(frame)
    }

    private suspend fun createSession(): ClientWebSocketSession {
        val url = "ws://127.0.0.1:${server.resolvedConnectors().first().port}/"
        return client.webSocketRawSession { url(url) }
    }
}

When providing files via a static block, a non-standard content-type is used.

Code:

fun Application.main() {
    install(Routing)
    
    static("assets") {
        files("public")
    }
}

File structure:

\- public
  |- some-html-file.html
  |- some-image.png
  \- some-ttf-font.ttf

Making requests to the following urls returns a response with the expected Content-Type header:

• /assets/some-html-file.html returns Content-Type: text/hml
• /assets/some-image.png returns Content-Type: image/png

However, making a request to /assets/some-ttf-font.ttf returns an unexpected Content-Type.

Example response:

HTTP/1.1 200 OK
Date: Thu, 08 Dec 2022 21:23:03 GMT
Server: Ktor/2.0.3
Last-Modified: Mon, 15 Aug 2022 22:09:01 GMT
Accept-Ranges: bytes
Content-Length: 335196
Content-Type: application/x-font-ttf

According to the IANA, the correct MIME type for a .ttf file is font/ttf.

Here is a list of file types which return an incorrect Content-Type:

• ttf
  • Expected: font/ttf
  • Actual: application/x-font-ttf
• woff2
  • Expected: font/woff2
  • Actual: application/octet-stream
• woff
  • Expected: font/woff
  • Actual: application/x-font-woff
• otf
  • Expected: font/otf
  • Actual: application/x-font-otf
• gz
  • Expected: application/gzip
  • Actual: application/x-compressed

The source of the issues is the Mimes.kt file in ktor-http module.

I want to avoid logging static file requests，currently I avoid logging static file requests like this:

  install(CallLogging) {
      level = Level.INFO
      filter { !it.parameters.contains("static-content-path-parameter") }
  }

Since the pathParameterName property of StaticContent.kt is private, I have to manually copy the String. Maybe we can make the pathParameterName String property of StaticContent.kt public ，or add configuration items in the CallLogging plugin.

https://jbs.zendesk.com/agent/tickets/4182169

My Environment

iosX64 Native - iOS 16.0 Simulator
Kotlin: 1.8.0
XCode: 14.0.1
Ktor Version: 2.2.3

Background

I'm running a Ktor server for mocking purposes on an iOS simulator. The problem is that when killing the application through XCode, I get no callback to stop the server gracefully (i.e. it's a SIGKILL).

If a request was in progress when the app is killed, it leads to the following exception upon restarting the app (unless I wait about 30 seconds).

io.ktor.utils.io.errors.PosixException.AddressAlreadyInUseException: EADDRINUSE

Feature Request

(I'm a mobile engineer so apologies if what I'm asking is incorrect). Can there be a flag added to the embeddedServer to allow port reuse, so that killing and restarting the app with the same server port does not lead to a crash?

I.e exposing the API used here to the embeddedServer configuration: https://youtrack.jetbrains.com/issue/KTOR-4442/EADDRINUSE-on-socket-bind-on-Linux.

Minimal Reproducible Example

https://github.com/SamCosta1/ktor-connection-bug-example/tree/ktor-killing-app-example

Notes

I've not experienced this issue running on Android, this seems to be Native only.

We would like to provide a solution for loading multiple configuration files, i.e. java -jar sample-app.jar -config=application.conf -config=applicationFoo.conf -config=applicationBar.conf.

The mechanism should work from left to right - it means if application.conf and applicationFoo.conf have common properties then values from application.conf should be overwritten by values from applicationFoo.conf, like shown below:

application.conf

ktor {
    deployment {
        port = 8080
    }

    application {
        class = <org.company.ApplicationClass>
    }
}

applicationFoo.conf

ktor {
    deployment {
        port = 8085
        testProperty = "TestValue"
    }
}

config {
    configurationProperty = "Config value"
}

result configuration:

ktor {
    deployment {
        port = 8085
        testProperty = "TestValue"
    }
    
    application {
        class = <org.company.ApplicationClass>
    }
}

config {
    configurationProperty = "Config value"
}

This used to work in 2.2.3:

fun main() {
  embeddedServer(Netty, port = 8082, host = "0.0.0.0", module = Application::module)
    .start(wait = true)
}

fun Application.module() {
  install(ContentNegotiation) {
    json()
  }
  routing {
    get {
      call.respond(value())
    }
  }
}

// fun value(): Data = Data("bar") works
fun value(): Any = Data("bar")

@Serializable
data class Data(val foo: String)

on 2.2.4 ktor throws

2023-03-21 01:45:13.069 [eventLoopGroupProxy-4-1] ERROR ktor.application - Unhandled: GET - /
kotlinx.serialization.SerializationException: Serializer for class 'Any' is not found.
Mark the class as @Serializable or provide the serializer explicitly.

so I have a simplest ktor app

fun main() {
    val env = applicationEngineEnvironment {
        module {
            routing {
                get("/") {
                    call.respondText("Hello, world!")
                }
            }
        }

        connector {
            this.port = 8080
        }
    }
    embeddedServer(CIO, env).start(wait = true)
}

    implementation("io.ktor:ktor-server-core:2.2.3")
    implementation("io.ktor:ktor-server-cio:2.2.3")

if you perform a scan of open tcp ports like this nmap localhost -Pn -p8080, the app crashes with

Exception in thread "DefaultDispatcher-worker-2" java.net.SocketException: Invalid argument
        at java.base/sun.nio.ch.Net.setIntOption0(Native Method)
        at java.base/sun.nio.ch.Net.setSocketOption(Net.java:455)
        at java.base/sun.nio.ch.Net.setSocketOption(Net.java:393)
        at java.base/sun.nio.ch.SocketChannelImpl.setOption(SocketChannelImpl.java:280)
        at io.ktor.network.sockets.ServerSocketImpl.accepted(ServerSocketImpl.kt:51)
        at io.ktor.network.sockets.ServerSocketImpl.acceptSuspend(ServerSocketImpl.kt:42)
        at io.ktor.network.sockets.ServerSocketImpl.access$acceptSuspend(ServerSocketImpl.kt:12)
        at io.ktor.network.sockets.ServerSocketImpl$acceptSuspend$1.invokeSuspend(ServerSocketImpl.kt)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
        Suppressed: kotlinx.coroutines.DiagnosticCoroutineContextException: [LazyStandaloneCoroutine{Cancelling}@23a3bd4f, Dispatchers.IO]

Hi,

I am trying to integrate with a service using OAuth2, however the response that the service gives does not get handled correctly by ktor. From the Asana website, this is a typical response given (https://developers.asana.com/docs/oauth)

{
  "access_token": "f6ds7fdsa69ags7ag9sd5a",
  "expires_in": 3600,
  "token_type": "bearer",
  "data": {
    "id": 4673218951,
    "gid": "4673218951",
    "name": "Greg Sanchez",
    "email": "gsanchez@example.com"
  },
  "refresh_token": "hjkl325hjkl4325hj4kl32fjds"
}

Currently ktor expects the response to only contain primitives. When trying to get the response after exchanging the code for the token, I get this error message from ktor.

Element class kotlinx.serialization.json.JsonObject is not a JsonPrimitive

I was wondering if this could be supported, possibly by adding these as extra parameters

Thanks!

Ktor version: 2.2.4

Kotlin version: 1.8.10

When running ktor server like:

val server = embeddedServer(Netty, port = 28080) {
        install(CallLogging) {
            callIdMdc()
        }
        routing {
            get("/test") {
                this.call.respondText("Hello, world!")
            }
            get("/err") {
                throw Exception("Unexpected")
            }
        }
    }

It will not log calls to /err route:

    @Test
    fun testRoot() = runTest {
        server.start(false)

        val client = HttpClient(CIO)
        assertEquals(200, client.get("http://localhost:28080/test").status.value)
        assertEquals(500, client.get("http://localhost:28080/err").status.value)

        client.close()
        server.stop() 
   }

Produced logs:

2023-03-16 10:14:44,668 INFO [] Application started in 0.284 seconds.                                            [ktor.application] [oroutine#2]
2023-03-16 10:14:44,669 INFO [] Application started: io.ktor.server.application.Application@4f3356c0             [ktor.application] [oroutine#2]
2023-03-16 10:14:45,043 INFO [] Responding at http://0.0.0.0:28080                                               [ktor.application] [oroutine#1]
2023-03-16 10:14:45,641 INFO [] 200 OK: GET - /test in 163ms                                     [ktor.application] [handler#16]
2023-03-16 10:14:45,669 ERROR [] Unhandled: GET - /err                                                            [ktor.application] [handler#29]
java.lang.Exception: Unexpected
	(Coroutine boundary)
	at io.ktor.util.pipeline.PipelineKt$execute$2.invokeSuspend(Pipeline.kt:478)
	at io.ktor.server.routing.Routing.executeResult(Routing.kt:188)
2023-03-16 10:14:47,492 INFO [] Application stopping: io.ktor.server.application.Application@4f3356c0            [ktor.application] [oroutine#2]
2023-03-16 10:14:47,493 INFO [] Application stopped: io.ktor.server.application.Application@4f3356c0             [ktor.application] [oroutine#2]

When CallLogin plug is not configured with mds, it will produces logs:

2023-03-16 10:13:47,070 INFO [] Application started in 0.269 seconds.                                            [ktor.application] [oroutine#2]
2023-03-16 10:13:47,071 INFO [] Application started: io.ktor.server.application.Application@6ac45c0c             [ktor.application] [oroutine#2]
2023-03-16 10:13:47,432 INFO [] Responding at http://0.0.0.0:28080                                               [ktor.application] [oroutine#1]
2023-03-16 10:13:48,031 ERROR [] Unhandled: GET - /err                                                            [ktor.application] [handler#29]
java.lang.Exception: Unexpected
	(Coroutine boundary)
	at io.ktor.util.pipeline.PipelineKt$execute$2.invokeSuspend(Pipeline.kt:478)
	at io.ktor.server.routing.Routing.executeResult(Routing.kt:188)
2023-03-16 10:13:48,170 INFO [] 200 OK: GET - /test in 312ms                                     [ktor.application] [handler#16]
2023-03-16 10:13:48,717 INFO [] 500 Internal Server Error: GET - /err in 730ms                   [ktor.application] [handler#29]
2023-03-16 10:13:49,761 INFO [] Application stopping: io.ktor.server.application.Application@6ac45c0c            [ktor.application] [oroutine#2]
2023-03-16 10:13:49,762 INFO [] Application stopped: io.ktor.server.application.Application@6ac45c0c             [ktor.application] [oroutine#2]

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/233

I want Ktor to drive my website. I have a bunch of HTML files but I want to strip the HTML suffix from those files when they are being served as the HTML suffix is just a technical detail that I want to strip.

When serving static files in static routes using the files(dir) DSL, it would be nice if a HEAD mapping was setup in addition to the GET one, in order to respect HTTP semantics.

The HEAD mapping should return (among others) the correct status code (e.g. 404 if the file doesn't exist) and also the file size as Content-Length for instance.

I stumbled upon this issue when trying to serve my local .m2 directory as static files to emulate a remote server, and Ivy (and likely Maven and Gradle) makes use of HEAD requests.

I had this simple route, serving the content of the assets resource folder when getting assets/*

static("assets") {
    resources("assets")
}

I realized that Ktor was responding to these requests with a weird Content-Type for files ending in .map like *.js.map (or *.css.map I guess).
This doesn't cause any real issue as most clients getting this kind of files ignore content type entirely. Still, I wanted to change that content type to something a little more standard like application/octet-stream or even something less standard but more useful for debugging, application/json.

So I tried to change this content type and couldn't find a way. I tried with an interceptor but you're somehow not allowed to change the content type header.
I ended up having to drop the use of static/resources. So now I have this:

    val oneYearInSeconds = Duration.ofDays(365).toSeconds()
    val pathParameterName = "path"
    get("/assets/{$pathParameterName...}") {
        val relativePath = call.parameters
            .getAll(pathParameterName)
            ?.joinToString(File.separator)
            ?: return@get

        val content = call.resolveResource(relativePath, "assets") { extension ->
            when {
                extension.endsWith(".js.map") -> ContentType.Application.Json
                else -> ContentType.defaultForFileExtension(extension)
            }
        } ?: return@get

        call.response.header(HttpHeaders.CacheControl, "public, max-age=$oneYearInSeconds, immutable")
        call.respond(content)
    }

Which is kinda longer and unfortunately doesn't use what's already there for static resources.

(I also implemented caching there, because 1. there does not seem to be caching enabled by default for static content and 2. Ktor's caching headers feature kinda sucks, but that's a different story)

Using the code found in the micrometer-metrics code snippet, and adding the dependencies for the dropwizard-metrics, if the plugin installation block for the Dropwizard plugin is added after the MicrometerMetrics plugin:

    install(MicrometerMetrics) {
        registry = PrometheusMeterRegistry(PrometheusConfig.DEFAULT)
        distributionStatisticConfig = DistributionStatisticConfig.Builder()
            .percentilesHistogram(true)
            .maximumExpectedValue(Duration.ofSeconds(20).toNanos().toDouble())
            .serviceLevelObjectives(
                Duration.ofMillis(100).toNanos().toDouble(),
                Duration.ofMillis(500).toNanos().toDouble()
            )
            .build()
        meterBinders = listOf(
            ClassLoaderMetrics(),
            JvmMemoryMetrics(),
            JvmGcMetrics(),
            ProcessorMetrics(),
            JvmThreadMetrics(),
            FileDescriptorMetrics(),
            UptimeMetrics()
        )
    }
    install(DropwizardMetrics) {
        JmxReporter.forRegistry(registry)
            .convertRatesTo(TimeUnit.SECONDS)
            .convertDurationsTo(TimeUnit.MILLISECONDS)
            .build()
            .start()
    }

then when the /metrics endpoint is scrapped, an exception is thrown;

java.lang.ClassCastException: class io.ktor.server.metrics.dropwizard.CallMeasure cannot be cast to class io.ktor.server.metrics.micrometer.CallMeasure (io.ktor.server.metrics.dropwizard.CallMeasure and io.ktor.server.metrics.micrometer.CallMeasure are in unnamed module of loader 'app')
        at io.ktor.server.metrics.micrometer.MicrometerMetricsKt$MicrometerMetrics$2$4.invoke(MicrometerMetrics.kt:148)
        at io.ktor.server.metrics.micrometer.MicrometerMetricsKt$MicrometerMetrics$2$4.invoke(MicrometerMetrics.kt:146)
        at io.ktor.server.application.hooks.ResponseSent$install$1.invokeSuspend(CommonHooks.kt:110)
        at io.ktor.server.application.hooks.ResponseSent$install$1.invoke(CommonHooks.kt)
        at io.ktor.server.application.hooks.ResponseSent$install$1.invoke(CommonHooks.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(SuspendFunctionGun.kt:88)
        at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invokeSuspend(DefaultTransform.kt:29)
        at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
        at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
        at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
        at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
        at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
        at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
        at io.ktor.server.engine.DefaultEnginePipelineKt.tryRespondError(DefaultEnginePipeline.kt:122)
        at io.ktor.server.engine.DefaultEnginePipelineKt.handleFailure(DefaultEnginePipeline.kt:54)
        at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invokeSuspend(DefaultEnginePipeline.kt:40)

A workaround appears to be registering the Dropwizard plugin before the MicrometerMetrics plugin

It is now not possible to serve static files from resources in GraalVM native image with default configuration:

routing {
    static("/assets") {
        resources("assets")
    }
}

It is because internal fun resourceClasspathResource supports only file, jar and jrt protocols.

But in native image, resources are supported by resource protocol.

PR: https://github.com/ktorio/ktor/pull/3388

Module: ktor-server-auth-jvm-2.2.2.jar
Class: io/ktor/server/auth/SessionAuth.kt:123

This line of code does not work for unauthorized users:
session<UserSession>("auth-session")

Since the default challengeFunction parameter is an empty implementation, users without a Session can also access resources protected by auth-session

{width=70%}

    install(Authentication) {
        form("auth-form") {
            validate { credentials ->
                if (credentials.name == "123" && credentials.password == "123") {
                    UserIdPrincipal(credentials.name)
                } else {
                    null
                }
            }
        }
        session<UserSession>("auth-session") // not work
    }

Steps to reproduce: serve static JS module files from a folder in a standard way like

routing {
    trace { application.log.trace(it.buildText()) }

    static("/scripts/") {
        this.staticRootFolder = File(rootPath)
        files(".")
    }
}

The problem: the MIME type for .mjs files is wrong and the browser cannot load the modules.

Error in the browser console:

Loading module from “http://localhost:81/scripts/asdf.mjs” was blocked because of a disallowed MIME type (“application/octet-stream”).

Expected result: ContentType header should be "text/javascript"

This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1781

Subsystem
Server, route building

Is your feature request related to a problem? Please describe.
Right now the accept() route selector only takes a single content type.

Describe the solution you'd like
It would be convenient if we could pass more than one, and it could match on any of the ones provided.

Motivation to include to ktor
That would reduce the amount of duplication when multiple content types are handled by the same handler.

If I install(ContentNegotiation) { gson() } and install(Mustache)
call.respond(MustacheContent("temp.hbs", mapOf("user" to sampleUser))) stops working correctly

Instead of returning proper html output, the server will return a json

{"template":"temp.hbs","model":{"user":{"id":1,"name":"John"}},"contentType":{"contentType":"text","contentSubtype":"html","content":"text/html","parameters":[{"name":"charset","value":"UTF-8","escapeValue":false}]}}

the return content type is application/json

After removing install(ContentNegotiation) { gson() } block, html is responded correctly but other api routes throw serialisation related errors as expected due to lack of gson

To reproduce send a text Message fragmented into 2 fragments, then Continuation Frame with FIN = false where there is nothing to continue, then an unfragmented Text Message, all sent in one chop.

Expected behavior: the connection is failed immediately since there is no message to continue.
Actual behavior: two text frames are sent back.

embeddedServer(CIO, host = "127.0.0.1", port = 0) {
    install(WebSockets)

    routing {
        webSocket("/") {
            for (frame in incoming) {
                outgoing.send(frame)
            }
        }
    }
}

I have written a simple websocket server with the following configuration:

install(WebSockets) {
    pingPeriod = Duration.ofSeconds(30)
}

Here is an excerpt of the logs:

2023-03-19 17:49:25.020 [DefaultDispatcher-worker-25] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: sending ping frame
2023-03-19 17:49:25.023 [DefaultDispatcher-worker-39] TRACE io.ktor.websocket.WebSocket - WebSocketSession(StandaloneCoroutine{Active}@5a55a4c1) receiving frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:49:25.023 [DefaultDispatcher-worker-39] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: received valid pong frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:49:25.023 [DefaultDispatcher-worker-39] TRACE io.ktor.websocket.WebSocket - WebSocket pinger has timed out
2023-03-19 17:49:55.028 [DefaultDispatcher-worker-5] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: sending ping frame
2023-03-19 17:49:55.029 [DefaultDispatcher-worker-52] TRACE io.ktor.websocket.WebSocket - WebSocketSession(StandaloneCoroutine{Active}@5a55a4c1) receiving frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:49:55.029 [DefaultDispatcher-worker-18] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: received valid pong frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:49:55.029 [DefaultDispatcher-worker-18] TRACE io.ktor.websocket.WebSocket - WebSocket pinger has timed out
2023-03-19 17:50:25.035 [DefaultDispatcher-worker-30] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: sending ping frame
2023-03-19 17:50:25.037 [DefaultDispatcher-worker-15] TRACE io.ktor.websocket.WebSocket - WebSocketSession(StandaloneCoroutine{Active}@5a55a4c1) receiving frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:50:25.038 [DefaultDispatcher-worker-15] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: received valid pong frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:50:25.038 [DefaultDispatcher-worker-15] TRACE io.ktor.websocket.WebSocket - WebSocket pinger has timed out

You can see that each time the server sends a PING frame, it immediately receives a PONG one, but each time we always get the following log:

TRACE io.ktor.websocket.WebSocket - WebSocket pinger has timed out

This is NOT true, the pong frame has been received in time (which is the default here, 15 seconds).

I found this in the code:

LOGGER.trace("WebSocket pinger has timed out")

if (rc == null) {
    // timeout
    // we were unable to send the ping or hadn't got a valid pong message in time,
    // so we are triggering close sequence (if already started then the following close frame could be ignored)

    onTimeout(CloseReason(CloseReason.Codes.INTERNAL_ERROR, "Ping timeout"))
    break
}

Shouldn't we put the log in the condition body? Currently, this is confusing.

TestApplicationEngine has a few runBlocking calls inside it.
This can cause trouble for advanced testing, when a suspend function is tested with a coroutineContext.

One notable example is Kotest. In order to use withClue, coroutineContext should contain errorCollectorContextElement. Otherwise context switching might result in exception. Right now there is no way to pass it into internal runBlocking calls of TestApplicationEngine, and it isn't passed from parent context.

Here's a sample code which reproduces the situation above:

@Test
fun test() = runBlocking(errorCollectorContextElement) {
        testApplication {
            withContext(Dispatchers.Unconfined) {
                withClue("some hint") {
                    withContext(Dispatchers.Unconfined) {
                        delay(2)
                        1 shouldBe 1
                    }
                }
            }
        }
    }

Kotest throws an IndexOutOfBoundsException here, because an error hint is popped from wrong errorCollector (one from another thread).

This particular error can be fixed on Kotest side, but I assume where might be more situations similar to this. Another example might be Exposed framework. One can create a wrapper transaction to rollback it after the test is finished, and context switching might result in losing context with the wrapper transaction.

Design defaults for the static content plugin to make it work out of the box.

https://sbom.lift.sonatype.com/report/T1-a0368c8f29fdaa555824-7e2f1cfcaa55a1-1677578344-d0731128c3504dc0ae64b539fb589112

Please use separate commit for every dependency

routing {
    route(Regex("""/abc/""")) {
        get(Regex("""cde""")) {
            call.respondText { "OK" }
        }
    }
}

None of the /abc/cde and /abc//cde is matched

routing {
    get(Regex("""\(?<a>\)""")) {
        call.respondText { "OK" }
    }
}

On request GET /<a>) there is IllegalArgumentException: No group with name <a>

get(Regex("""\/abc""")) should have the same behavior as get(Regex("""/abc"""))

Subsystem

Client/Server

Motivation

It is much more useful to compare status codes between each other. For example:

httpResponse.status < HttpStatusCode.OK

Currently, in case, if I want to compare status codes I should use next comparison:

httpResponse.status.value < HttpStatusCode.OK.value

Solution

In my PR HttpStatusCode will implements Comparable interface and realize compareTo with any other HttpStatusCode

There are two parameters that are used for shutdown configuration: gracePeriod and shutdownTimeout`. There are used in ShutdownHook and stop method

In KTOR-5359 configuration for ShutdownHook was added

Multiple problems related to private responses caching:

• private cache responses can be shared between multiple users when using HttpClient in a proxy use case
• s-maxage had a typo and was used for private responses, while should be used for shared
• only first Cache-Control header was checked, but they should be merged