Changelog 2.3 version
2.3.7
released 7th December 2023
Client
WebSockets: Confusing error message when server doesn't respond with Upgrade
Current error description:
io.ktor.client.call.NoTransformationFoundException: Expected response body of the type 'class io.ktor.client.plugins.websocket.DefaultClientWebSocketSession' but was 'class io.ktor.utils.io.ByteBufferChannel'
In response from `https://example.com`
Response status `500 Server Error`
Response header `ContentType: text/xml; charset=UTF-8`
Request header `Accept: application/xml`
Expected error description:
ServerResponseException with the 500 error and ideally the textual response
Server
ContentNegotiation: Adding charset to content type of JacksonConverter breaks request matching
When trying to get the ContentNegotiation to specify the charset on JSON responses like so:
install(ContentNegotiation) {
jackson(contentType = ContentType.Application.Json.withCharset(Charsets.UTF_8))
}
the matching on ingoing requests breaks.
When looking at RequestConverter
this is no surprise, since it strips the charset from the requestContentType
and then tries to match the full content type provided at registration.
val requestContentType = try {
call.request.contentType().withoutParameters()
}
...
if (!requestContentType.match(registration.contentType)) {
LOGGER.trace(
"Skipping content converter for request type ${receiveType.type} because " +
"content type $requestContentType does not match ${registration.contentType}"
)
return null
}
I guess the charset should also be stripped from the registration content type or there should be another way to specify the charset parameter for responses which does not affect the matching for requests.
Server ContentNegotiation no longer allows multiple decoders for one Content-Type
In Ktor Server 2.1.3, it was possible to have multiple ContentNegotiation "codecs" installed for one Content-Type. During request decoding, the codecs were tried sequentially and the first non-null result was used. In Ktor 2.2.1, this no longer works - only the first codec is tried. This causes issues for example when using org.json
and Kotlinx.serialization in parallel (I'm using a custom shim converter that allows me to directly receive JSONObject
s).
I suppose the problem appeared in commit https://github.com/ktorio/ktor/commit/be329698362b9c12ca43b5969bdbbcbdb3496081. I don't know the exact mechanism at play here, but I think there is a subtle logic change in the reimplemented convertBody
function or its caller.
This demo reproduces the problem:
- Application.kt:
package com.example
import io.ktor.http.ContentType
import io.ktor.server.application.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import io.ktor.server.plugins.contentnegotiation.ContentNegotiation
import io.ktor.server.request.receive
import io.ktor.server.response.respondText
import io.ktor.server.routing.post
import io.ktor.server.routing.routing
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0", module = Application::module)
.start(wait = true)
}
fun Application.module() {
install(ContentNegotiation) {
cvtA(ContentType.Application.Json)
cvtB(ContentType.Application.Json)
}
routing {
post("/") {
val request = call.receive<CvtBResult>()
call.respondText("ECHO: " + request.data)
}
}
}
- CvtA.kt:
package com.example
import io.ktor.http.ContentType
import io.ktor.http.content.OutgoingContent
import io.ktor.http.content.TextContent
import io.ktor.http.withCharsetIfNeeded
import io.ktor.serialization.Configuration
import io.ktor.serialization.ContentConverter
import io.ktor.util.reflect.TypeInfo
import io.ktor.utils.io.ByteReadChannel
import io.ktor.utils.io.charsets.Charset
import io.ktor.utils.io.jvm.javaio.toInputStream
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
data class CvtAResult(val data: String)
class CvtA : ContentConverter {
override suspend fun deserialize(charset: Charset, typeInfo: TypeInfo, content: ByteReadChannel): Any? {
return if (typeInfo.type == CvtAResult::class) {
withContext(Dispatchers.IO) {
val reader = content.toInputStream().reader(charset)
CvtAResult(reader.readText())
}
} else {
null
}
}
override suspend fun serializeNullable(
contentType: ContentType,
charset: Charset,
typeInfo: TypeInfo,
value: Any?
): OutgoingContent? {
return if (value != null && typeInfo.type == CvtAResult::class) {
TextContent((value as CvtAResult).data, contentType.withCharsetIfNeeded(charset))
} else {
null
}
}
}
/**
* Register a converter for org.JSON objects.
*/
fun Configuration.cvtA(
contentType: ContentType = ContentType.Application.Json,
) {
register(contentType, CvtA())
}
- CvtB:kt:
package com.example
import io.ktor.http.ContentType
import io.ktor.http.content.OutgoingContent
import io.ktor.http.content.TextContent
import io.ktor.http.withCharsetIfNeeded
import io.ktor.serialization.Configuration
import io.ktor.serialization.ContentConverter
import io.ktor.util.reflect.TypeInfo
import io.ktor.utils.io.ByteReadChannel
import io.ktor.utils.io.charsets.Charset
import io.ktor.utils.io.jvm.javaio.toInputStream
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
data class CvtBResult(val data: String)
class CvtB : ContentConverter {
override suspend fun deserialize(charset: Charset, typeInfo: TypeInfo, content: ByteReadChannel): Any? {
return if (typeInfo.type == CvtBResult::class) {
withContext(Dispatchers.IO) {
val reader = content.toInputStream().reader(charset)
CvtBResult(reader.readText())
}
} else {
null
}
}
override suspend fun serializeNullable(
contentType: ContentType,
charset: Charset,
typeInfo: TypeInfo,
value: Any?
): OutgoingContent? {
return if (value != null && typeInfo.type == CvtBResult::class) {
TextContent((value as CvtBResult).data, contentType.withCharsetIfNeeded(charset))
} else {
null
}
}
}
/**
* Register a converter for org.JSON objects.
*/
fun Configuration.cvtB(
contentType: ContentType = ContentType.Application.Json,
) {
register(contentType, CvtB())
}
Test request can be made via curl:
curl -i http://localhost:8080/ -H "Content-Type: application/json" --data '{}'
I expected the following output (behaviour of Ktor 2.1.3):
HTTP/1.1 200 OK
Content-Length: 8
Content-Type: text/plain; charset=UTF-8
ECHO: {}
However, with Ktor 2.2.1 this is printed:
HTTP/1.1 415 Unsupported Media Type
Content-Length: 0
As for the parallel use of org.json and kotlinx.serialization: I've previously used org.json only and then introduced kotlinx.serialization for handling data with static schema. I've now noticed that kotlinx.serialization.json provides JsonElement which appears to be able to handle data without a fixed schema. I might therefore switch away from using org.json entirely and thus this issue wouldn't affect me anymore.
High Native Server Memory Usage
I have a "hello, world" Ktor native server (Kotlin/Native on Linux) and have noticed very high memory usage under heavy load. While memory usage starts around 30MB, it climbs to around 9GB when the server is saturated with requests. Memory does fall back to around 125MB after requests have ceased. I've tried different allocators and options but don't see much difference.
Repo for reproducing:
https://github.com/jamesward/ktor-native-server/tree/mem_repro
ApacheBench for request saturation:
ab -c 8 -n 1000000 http://localhost:8080/
I know it's an apples-to-oranges comparison, but running the same thing on the JVM doesn't seem to cause any memory spiking.
I'm not sure if there are optimizations in Ktor for this use case or if it is all on Kotlin/Native.
2.3.6
released 7th November 2023
Build System Plugin
Outdated Gradle jib plubin does not support application/vnd.oci.image.index.v1+json media type
If you try to use a gcr.io image as a base image for containerization, you will get the following error:
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':jibDockerBuild'.
> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: Tried to pull image manifest for gcr.io/distroless/java17-debian11:debug but failed because: Manifest with tag 'debug' has media type 'application/vnd.oci.image.index.v1+json', but client accepts 'application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.v2+json,application/vnd.docker.distribution.manifest.v1+json,application/vnd.docker.distribution.manifest.list.v2+json'.
Adding the latest version of jib plugin explicitly solves the problem.
How to reproduce:
- generate fresh Ktor project at https://start.ktor.io/ (Ktor 2.3.4 as for now)
- try to
./gradlew -Djib.from.image='gcr.io/distroless/java17-debian11:debug' jibDockerBuild
and you will get the above error
Workaround:
Just add jib plugin explicitly.
plugins {
kotlin("jvm") version "1.9.10"
id("io.ktor.plugin") version "2.3.4"
id("com.google.cloud.tools.jib") version "3.3.2" // <<------------
}
Client
Darwin: EOFException when sending multipart data using Ktor 2.3.4
After bumping ktor client version from 2.3.3 to 2.3.4 we are getting a "Optional(io.ktor.utils.io.errors.EOFException: Premature end of stream: expected 467 bytes)) error" on iOS (works fine on Android) when sending a form with binary data.
The code simplified looks like this (image is a byte array):
val formData = formData {
append(
key = "image",
value = image,
headers = Headers.build {
append(HttpHeaders.ContentType, "image/jpeg")
append(HttpHeaders.ContentDisposition, "filename=image.jpeg")
},
)
}
return httpClient.submitFormWithBinaryData(url, formData) {}.body()
We didn't change anything on our side and downgrading to 2.3.3 fixes the issue, so it seems to be related to the update.
AndroidClientEngine cannot handle content length that exceeds Int range
In execute()
method in AndroidClientEngine
class, following line makes it cannot handle content length that exceeds integer range.
contentLength?.let { setFixedLengthStreamingMode(it.toInt()) } ?: setChunkedStreamingMode(0)
Since there is setFixedLengthStreamingMode(long)
, removing toInt()
call may fix the issue.
Darwin: Even a coroutine Job is canceled network load keeps high
I guess the problem is caused by DarwinSession.close:
override fun close() {
if (!closed.compareAndSet(false, true)) return
session.finishTasksAndInvalidate()
}
In Apple's document,
This method returns immediately without waiting for tasks to finish. Once a session is invalidated, new tasks cannot be created in the session, but existing tasks continue until completion . After the last task finishes and the session makes the last delegate call related to those tasks, the session calls the URLSession:didBecomeInvalidWithError:
method on its delegate, then breaks references to the delegate and callback objects. After invalidation, session objects cannot be reused.
To cancel all outstanding tasks, call invalidateAndCancel
instead.
So, in close function, invoking invalidateAndCancel is better than finishTasksAndInvalidate
Ktor JS client unconfigurable logging in node
Many ktor client plugins contain their own LOGGER, e.g
internal val LOGGER = KtorSimpleLogger("io.ktor.client.plugins.websocket.WebSockets")
Which can't be configured.
Implementation of KtorSimpleLoggerJs.kt
@Suppress("FunctionName")
public actual fun KtorSimpleLogger(name: String): Logger = object : Logger {
override fun error(message: String) {
console.error(message)
}
override fun error(message: String, cause: Throwable) {
console.error("$message, cause: $cause")
}
override fun warn(message: String) {
console.warn(message)
}
override fun warn(message: String, cause: Throwable) {
console.warn("$message, cause: $cause")
}
override fun info(message: String) {
console.info(message)
}
override fun info(message: String, cause: Throwable) {
console.info("$message, cause: $cause")
}
override fun debug(message: String) {
console.debug("DEBUG: $message")
}
override fun debug(message: String, cause: Throwable) {
console.debug("DEBUG: $message, cause: $cause")
}
override fun trace(message: String) {
console.debug("TRACE: $message")
}
override fun trace(message: String, cause: Throwable) {
console.debug("TRACE: $message, cause: $cause")
}
}
It floods logs with unnecessary and useless messages on Node as console.debug is just an alias for console.log
TRACE: Sending WebSocket request [object Object]
This logger should be configurable with Logging plugin.
"Server sent a subprotocol but none was requested" when using Node WebSockets
The following code fails with "Server sent a subprotocol but none was requested":
val ktorClient = HttpClient(Js) {
install(WebSockets)
}
val url = Url("wss://apollo-fullstack-tutorial.herokuapp.com/graphql")
val connection = ktorClient.request<DefaultClientWebSocketSession>(url) {
headers {
append("Sec-WebSocket-Protocol", "graphql-ws")
}
}
This seems to happen because protocols
is never passed to the WebSocket
constructor (here)
Client unable to make subsequent requests after the network disconnection and connection when ResponseObserver is installed
Description:
When using the Ktor HTTP client library to download files, an issue has been observed where the client fails to recover after an interrupted download caused by a loss of internet connection. This behavior is particularly evident when the provided code snippet is used to perform the download.
Steps to Reproduce:
- Establish a working internet connection.
- Execute the provided code snippet for making an HTTP request to download a file using the Ktor HTTP client.
- During the file download process, intentionally disrupt the internet connection.
- Allow the client to handle the connection loss and observe its behavior.
Expected Behavior: Upon detecting a loss of connectivity during the file download process, the Ktor HTTP client should throw an appropriate exception (e.g., a network-related exception) to indicate the connection failure. Subsequent attempts to make new HTTP requests using the client, after restoring the internet connection, should work as expected. The client should no longer be stuck in an unrecoverable state and should be capable of handling further requests without hindrance.
Actual Behavior: Currently, when the internet connection is disrupted during the file download process, the Ktor HTTP client appears to enter an unrecoverable state. Subsequent attempts to use the client to make additional HTTP requests (even after the internet connection is restored) fail to complete successfully. The client seems to be stuck in a state where it cannot recover from the interrupted download.
Code Snippet:
try {
val httpResponse: HttpResponse = httpClient.get(url) {
onDownload { bytesSentTotal, contentLength ->
// Progress tracking logic can be added here if needed
}
}
val responseBody: ByteArray = httpResponse.body()
file.writeBytes (responseBody)
} catch (e: Exception) {
// Exception handling logic can be added here
}
Exception Stack Trace:
java.net.SocketTimeoutException: timeout at com.android.okhttp.okio.Okio$3.newTimeoutException(Okio.java:214) at com.android.okhttp.okio.AsyncTimeout.exit(AsyncTimeout.java:263) at com.android.okhttp.okio.AsyncTimeout$2.read(AsyncTimeout.java:217) at com.android.okhttp.okio.RealBufferedSource.read(RealBufferedSource.java:51) at com.android.okhttp.internal.http.Http1xStream$FixedLengthSource.read(Http1xStream.java:395) at com.android.okhttp.okio.RealBufferedSource$1.read(RealBufferedSource.java:372) at java.io.BufferedInputStream.fill(BufferedInputStream.java:248) at java.io.BufferedInputStream.read1(BufferedInputStream.java:288) at java.io.BufferedInputStream.read(BufferedInputStream.java:347) at io.ktor.utils.io.jvm.javaio.ReadingKt$toByteReadChannel$1.invokeSuspend(Reading.kt:55) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684) Caused by: java.net.SocketException: Socket closed at java.net.SocketInputStream.read(SocketInputStream.java:188) at java.net.SocketInputStream.read(SocketInputStream.java:143) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.readFromSocket(ConscryptEngineSocket.java:945) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.processDataFromSocket(ConscryptEngineSocket.java:909) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.readUntilDataAvailable(ConscryptEngineSocket.java:824) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.read(ConscryptEngineSocket.java:797) at com.android.okhttp.okio.Okio$2.read(Okio.java:138) at com.android.okhttp.okio.AsyncTimeout$2.read(AsyncTimeout.java:213) at com.android.okhttp.okio.RealBufferedSource.read(RealBufferedSource.java:51) at com.android.okhttp.internal.http.Http1xStream$FixedLengthSource.read(Http1xStream.java:395) at com.android.okhttp.okio.RealBufferedSource$1.read(RealBufferedSource.java:372) at java.io.BufferedInputStream.fill(BufferedInputStream.java:248) at java.io.BufferedInputStream.read1(BufferedInputStream.java:288) at java.io.BufferedInputStream.read(BufferedInputStream.java:347) at io.ktor.utils.io.jvm.javaio.ReadingKt$toByteReadChannel$1.invokeSuspend(Reading.kt:55) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
Note:
The preceding stack trace remains consistent for both the unsuccessful download request and the subsequent requests that follow it.
Environment:
- Ktor version: [2.3.3]
- HttpClientEngine: Android
- Operating System: [Android 11]
WebSockets (CIO): Connection Failure Due to Lowercase 'upgrade' in 'Connection: upgrade' Header
Hello,
I have encountered an issue while trying to establish a WebSocket connection using Ktor. It appears that the 'Connection: upgrade' header sent by Ktor has 'upgrade' in lowercase, which is causing the WebSocket connection to fail.
In my application scenario, I tried CIO, Java, and OkHttp engines and captured packets for analysis. Only the OkHttp engine used the header with capital letters when establishing a connection, that is, "Connection: Upgrade", and then the connection was successful, while the other both engines use lowercase "upgrade", which causes the connection to fail.
By reading the code, I found that Ktor used lowercase "upgrade" here. I believe adjusting the capitalization to 'Connection: Upgrade' may resolve the WebSocket connection issue. I hope this information is helpful and look forward to hearing back on whether this could be addressed in a future release of Ktor.
Thank you.
WinHttp: ArrayIndexOutOfBoundsException when sending WS frame with empty body
It appears that sending a frame with an empty body (e.g. a Frame.Text("")
) crashes an internal coroutine that fails incoming
channel collections.
The root cause is the following:
Caused by: kotlin.ArrayIndexOutOfBoundsException
at 0 ??? 7ff7ae9c8d62 kfun:kotlin.Throwable#<init>(){} + 98
at 1 ??? 7ff7ae9c2f7f kfun:kotlin.Exception#<init>(){} + 79
at 2 ??? 7ff7ae9c31ef kfun:kotlin.RuntimeException#<init>(){} + 79
at 3 ??? 7ff7ae9c337f kfun:kotlin.IndexOutOfBoundsException#<init>(){} + 79
at 4 ??? 7ff7ae9c3a4f kfun:kotlin.ArrayIndexOutOfBoundsException#<init>(){} + 79
at 5 ??? 7ff7ae9f1028 ThrowArrayIndexOutOfBoundsException + 120
at 6 ??? 7ff7aed892a7 Kotlin_Arrays_getByteArrayAddressOfElement + 23
at 7 ??? 7ff7ae9b8ed1 kfun:kotlinx.cinterop#addressOf__at__kotlinx.cinterop.Pinned<kotlin.ByteArray>(kotlin.Int){}kotlinx.cinterop.CPointer<kotlinx.cinterop.ByteVarOf<kotlin.Byte>> + 177
at 8 ??? 7ff7aed26b9d kfun:io.ktor.client.engine.winhttp.internal.WinHttpWebSocket.sendFrame#internal + 2877
at 9 ??? 7ff7aed255f4 kfun:io.ktor.client.engine.winhttp.internal.WinHttpWebSocket.$sendNextFrameCOROUTINE$0.invokeSuspend#internal + 2580
at 10 ??? 7ff7ae9ce785 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 1029
It turns out the WinHttpWebSocket
tries to use buffer.addressOf(0)
even when the body (buffer
) is empty:
https://github.com/ktorio/ktor/blob/b34415dafca82090e408761788a9b7ffb24788d7/ktor-client/ktor-client-winhttp/windows/src/io/ktor/client/engine/winhttp/internal/WinHttpWebSocket.kt#L201
I think the same issue is probably true for receiving frames with empty bodies:
https://github.com/ktorio/ktor/blob/b34415dafca82090e408761788a9b7ffb24788d7/ktor-client/ktor-client-winhttp/windows/src/io/ktor/client/engine/winhttp/internal/WinHttpWebSocket.kt#L109
Note that it might be hard to notice due to https://youtrack.jetbrains.com/issue/KT-62794 (we need to use printStackTrace()
to see the cause).
Reproducer:
fun main() {
runBlocking {
val http = HttpClient(WinHttp) {
install(WebSockets)
}
val session = http.webSocketSession("wss://ws.postman-echo.com/raw")
try {
session.send(Frame.Text(""))
// This fails with ArrayIndexOutOfBoundsException, not because of the received echoed frame,
// but because of the failed send causing the incoming channel to fail (see sendFrame in the stacktrace)
session.incoming.receive()
} finally {
session.close()
}
}
}
Docs
The generator adds the '-jvm' suffix to all dependencies
Generator
The newly generated project is having problems with gradle
I think We have the same bug
{width=70%}
After updating ktor plugin (for gradle) to version 2.3.6, we also updated jackson-core-2.15.2 . But it has some problems with gradle 7.5.1 .
With the ktor plugin 2.3.5, it works fine
Because of this, the newly created project does not build in ide
Server
CIO: getEngineHeaderValues() returns duplicated values
Description:
When using embeddedCIOServer and geting all headers with call.response.headers.allValues()
returned values are not as expected (examples below). When switching to another engine(e.g. Netty) returned values are as expected.
Describe the bug:
Function allValues()
uses abstract function getEngineHeaderValues(name: String)
from io.ktor.server.cio.CIOApplicationResponse
and its implementation in cio-server is pretty confusing and in my testing wrong. I'm setting custom header for a route response like this:
call.response.headers.append("Test-Header", "Test-Value")
when getting header values the output is
{Test-Header=[Test-Value, Test-Value, Test-Value]}
When adding additional header with the same name:
call.response.headers.append("Test-Header", "Test-Value2")
the output is:
{Test-Header=[Test-Value, Test-Value, Test-Value, Test-Value2, Test-Value, Test-Value, Test-Value, Test-Value2]}
Output when using netty server:
{Test-Header=[Test-Value]}
{Test-Header=[Test-Value, Test-Value2, Test-Value, Test-Value2]}
*Second output is also correct since there are 2 headers with 2 values but with the same key
To reproduce:
Create simple CIO and Netty servers, add headers on the route and get response headers via allValues()
function.
Or clone the project https://github.com/Theanko1412/ktor-issue-demo
Additional information:
After looking at git history this function was changed in commit from 2017, I mocked old behavior of this function with allValuesFixed()
in my demo project and it is working as expected.
The response is having correct values, problem is only in retrieving it with allValues()
Proposed solution:
Revert function to its previous state.
If someone can provide some more information why was this changed and how was current implementation meant to be used I would appreciate it. Currently don't see any other way to correctly retrieve all headers from a response other than this.
Issue seems to be pretty old but couldn't find any old issues about this, if this is duplicate I would appreciate if you can point me to the solution and reasons why am I getting triple header value.
YAML properties with literal value null cannot be read since 2.3.1
Hello,
since version 2.3.1 (I suspect it came with the change https://youtrack.jetbrains.com/issue/KTOR-5797, but that's just a guess), it is not possible anymore to access a YAML configuration value that is assigned an explicit null value via the propertyOrNull
method.
Example:
ktor:
application:
modules:
- com.example.ApplicationKt.module
deployment:
port: 8080
sample:
prop1: null
# pro2: null
package com.example
import io.ktor.server.application.*
import io.ktor.server.response.*
import io.ktor.server.routing.*
fun main(args: Array<String>) {
io.ktor.server.netty.EngineMain.main(args)
}
fun Application.module() {
routing {
get("/") {
val config = call.application.environment.config
call.application.log.info("prop1: ${config.propertyOrNull("ktor.sample.prop1") == null}")
call.application.log.info("prop2: ${config.propertyOrNull("ktor.sample.prop2") == null}")
call.respondText("OK")
}
}
}
With up to version 2.3.0, it runs fine, but starting at 2.3.1, it fails as follows:
ApplicationConfigurationException: Expected primitive or list at path ktor.sample.prop1, but was class net.mamoe.yamlkt.YamlNull
I suspect that's the case because, in the YamlConfig#propertyOrNull
implementation the line val value = yaml[parts.last()] ?: return null
does not work as intended because the evaluation of the expression yaml[parts.last()]
yields an instance of YamlNull
instead of null
itself. Therefore, the early exit with the evlis operator ?: return null
does not fire. But again, that's just a suspicion of mine.
I am not sure if this is entirely intentional or if it indicates a bug. If it does, I would be glad if you could fix it. My temporary workaround is commenting out configuration variables whose values should be null instead of explicitly assigning null
because then it behaves as previously.
I have attached the repro example from above as zip.
Test Infrastructure
Resolved connectors job does not complete in TestApplicationEngine
BaseApplicationEngine launches job on application coroutine context which awaits resolvedConnectors to complete. They are completed successfully on start with NettyApplicationEngine but never completes with TestApplicationEngine.
In tests, we'd like to await all the async tasks that were started with code:
withTimeoutOrNull(timeout) { application.coroutineContext.job.children.forEach { scope -> scope.join() } } ?: throw TimeoutException("Timed out waiting ($timeout) for child coroutines to finish")
but the resolvedConnectors job is still awaiting resolved connectors.
As a workaround, the first thing we do (after engine initialization) is
engine.application.coroutineContext.job.children.firstOrNull()?.cancelAndJoin()
but this is hacky a little bit.
Other
KTor 2.3.5 Kotlin 1.9.x upgrade is a breaking change
Our project uses Kotlin 1.7. It's now unable to benefit from this "bugfix release", due to binary incompatibility.
e.g.
lass 'io.ktor.http.HttpHeaders' was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.9.0, expected version is 1.7.1.
2.3.5
released 5th October 2023
Client
300+ ktor-client-java threads eat up lots of memory
See attached fatal error log that was generated because IDEA failed to allocate a small amount of native memory (via malloc()
) on an M1 Max machine with 64G physical memory after 2 days of uptime. In the log, there are 300+ threads like this (see below) that alone eat up 2.5G+ memory just for their stacks. This does not look right and could be (but not definitely is) a reason for OOME:
0x000000014420b400 JavaThread "HttpClient-22-SelectorManager" daemon [_thread_in_native, id=64651, stack(0x00000002cae50000,0x00000002cb053000)]
0x0000000136207600 JavaThread "ktor-client-java-2" daemon [_thread_blocked, id=58891, stack(0x00000002cb05c000,0x00000002cb25f000)]
0x0000000142868e00 JavaThread "ktor-client-java-3" daemon [_thread_blocked, id=40967, stack(0x000000029d0a0000,0x000000029d2a3000)]
0x000000032aa29000 JavaThread "Netty Builtin Server 3" daemon [_thread_in_native, id=47419, stack(0x00000002ceffc000,0x00000002cf1ff000)]
0x0000000136e5c800 JavaThread "ktor-client-java-0" daemon [_thread_blocked, id=51255, stack(0x000000033f490000,0x000000033f693000)]
0x000000030bf58400 JavaThread "HttpClient-35-SelectorManager" daemon [_thread_in_native, id=165147, stack(0x000000033f69c000,0x000000033f89f000)]
0x000000030bf58a00 JavaThread "ktor-client-java-1" daemon [_thread_blocked, id=83379, stack(0x00000003595c0000,0x00000003597c3000)]
0x0000000147f23400 JavaThread "ktor-client-java-2" daemon [_thread_blocked, id=66847, stack(0x0000000364d18000,0x0000000364f1b000)]
0x0000000147e5e000 JavaThread "ktor-client-java-3" daemon [_thread_blocked, id=27415, stack(0x0000000365410000,0x0000000365613000)]
Apache5 engine limits concurrent requests to individual route to 5
When upgrading from Apache
to Apache5
we noticed bottlenecking on our HTTP requests. Digging through the Apache engine code we noticed that https://github.com/ktorio/ktor/blob/0d63c4d8d987098071527125e2bb4c11da9b2aa3/ktor-client/ktor-client-apache5/jvm/src/io/ktor/client/engine/apache5/Apache5Engine.kt#L94-L95 got typod. One of those is supposed to be setMaxConnPerRoute
.
DarwinClientEngine WebSocket rejects all received pongs
If I set a pingInterval
bigger than 0
on the DarwinClientEngine websocket it will invariably timeout because it will reject all received pongs as invalid.
[TRACE] (io.ktor.websocket.WebSocket): WebSocket Pinger: sending ping frame
[TRACE] (io.ktor.websocket.WebSocket): WebSocketSession(StandaloneCoroutine{Active}@f84385f0) receiving frame Frame PONG (fin=true, buffer len = 0)
[TRACE] (io.ktor.websocket.WebSocket): WebSocket Pinger: received invalid pong frame Frame PONG (fin=true, buffer len = 0), continue waiting
So I'm taking a look at PingPong.kt to see what validation is done
// wait for valid pong message
while (true) {
val msg = channel.receive()
if (String(msg.data, charset = Charsets.ISO_8859_1) == pingMessage) {
LOGGER.trace("WebSocket Pinger: received valid pong frame $msg")
break
}
LOGGER.trace("WebSocket Pinger: received invalid pong frame $msg, continue waiting")
}
So it expects the received pong to have the same payload as the outgoing ping. Let's take look at the DarwinWebsocketSession.kt and see what it does with the payload.
FrameType.PING -> {
task.sendPingWithPongReceiveHandler { error ->
if (error != null) {
cancel("Error receiving pong", DarwinHttpRequestException(error))
return@sendPingWithPongReceiveHandler
}
_incoming.trySend(Frame.Pong(ByteReadPacket.Empty))
}
}
Here we can see that it always forwards an empty pong since the Darwin API doesn't expose the payload when sending and receiving ping & pongs.
Since Darwin APIs doesn't support payloads when sending pings maybe there should be an option to configure the pinger without any validation on the payload? The way it's implemented now the pinger just doesn't work at all on Darwin platforms.
Docs
Verify the KMM tutorial and fix sample
Locally there is a version incompatibility with '2.3.5' when running the code sample. Furthermore, refer to the thread on Slack to verify the tutorial, specifically the setup for iOS.
Generator
Generator returns 500 status code for generate requests
Maven Ktor project fails to start application with an error: SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder"
To reproduce:
Create maven project
Try to run Ktor Application
Expected:
Application started successfully
Actual:
Error:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
You can fix it by adding dependency:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
2.3.4
released 1st September 2023
Build System Plugin
JRE_19 Not available for ktor docker plugin
The max JRE version in ktor docker plugin is JRE_17, but JRE_19 and JRE_20 are already out.
If possible, a suggested approach is to use a string type for that field.
Client
Darwin: App hangs when sending a huge MultiPart request without access to network
When device is in Airplane Mode and a request with MultiPartFormDataContent is started, the app gets stuck in an infinite loop (in OutgoingContent.toDataOrStream()
at line 52).
Sample request:
httpClient.post("...") {
setBody(
MultiPartFormDataContent(
formData {
append(
"file",
content,
Headers.build {
append(HttpHeaders.ContentType, "application/pdf")
append(HttpHeaders.ContentDisposition, "filename=\"file.pdf\"")
}
)
}
)
)
}
The problem in OutgoingContent.toDataOrStream()
is that outputStream.write
in my case returns -1
and thus the condition of the While loop (offset < read
) never becomes false.
Confusing NoTransformationFoundException
Some people new to Ktor cannot understand the cause of the NoTransformationFoundException
and how to fix the problem.
Cookie name-value pairs should be separated by a semicolon instead of a comma
Ktor Version and Engine Used (client or server and name)
2.2.3, OkHttp
Describe the bug
When sending Cookie
header with multiple name-value pairs, they are separated with a comma instead of a semicolon.
See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cookie#directives
To Reproduce
val response = client.post("http://localhost") {
cookie("firstCookie", "first")
}.body<HttpResponse>()
with the DefaultRequest plugin set up like this :
defaultRequest {
cookie("secondCookie", "second")
}
generates the following request :
curl -X POST "http://localhost" -H "Cookie:firstCookie=first,secondCookie=second"
Expected behavior
The Cookie
header should be:
Cookie: firstCookie=first; secondCookie=second
Remarks
- The function
cookie
handles multiple name-value pairs correctly - The issue happens when adding cookies as headers "manually", with
header(HttpHeaders.Cookie, "firstCookie=first")
- It also happens when using
cookie
in both the request builder and thedefaultRequest
builder. - This way, the
Cookie
header is created by the function mergeHeaders which uses a comma for every headers. - It has been changed to fix this issue about the
Accept
header
Digest Auth: algorithm isn't specified in the Authorization header
To reproduce use the following code for a client and server:
val client = HttpClient(CIO) {
install(Auth) {
digest {
algorithmName = "SHA-256"
credentials {
DigestAuthCredentials(username = "jetbrains", password = "foobar")
}
realm = "Access to the '/' path"
}
}
}
val r = client.get<String>("http://localhost:9090/secret")
println(r)
fun getMd5Digest(str: String): ByteArray = MessageDigest.getInstance("MD5").digest(str.toByteArray(UTF_8))
val myRealm = "Access to the '/' path"
val userTable: Map<String, ByteArray> = mapOf(
"jetbrains" to getMd5Digest("jetbrains:$myRealm:foobar"),
"admin" to getMd5Digest("admin:$myRealm:password")
)
embeddedServer(Netty, port = 9090) {
install(CORS) {
anyHost()
allowHeaders { true }
exposeHeader(HttpHeaders.WWWAuthenticate)
HttpMethod.DefaultMethods.forEach { method(it) }
}
install(Authentication) {
digest("auth-digest") {
algorithmName = "SHA-256"
realm = myRealm
digestProvider { userName, _ ->
userTable[userName]
}
}
}
routing {
authenticate("auth-digest") {
get("/secret") {
call.respondText { "Secret" }
}
}
}
}.start()
As a result, the 401
status code is return instead of expected 200
. It works as expected only if the default MD5
algorithm is used.
Docs
Discrepancies in Websockets tutorials
Several improvements can be made to the tutorials for creating a websocket chat:
- Server
- the dependencies generated by the plugin look different and do not include the
ch.qos.logback:logback-classic
- in source code the
routing
function is missing from the first appearance ofconfigureSockets
- feature requests need formatting
- remove the use of plural formatted keywords (ex.
Frame
s)
- Client
- screenshots are missing in Create a new project
The io.ktor.plugin for using docker is broken with version 2.3.4
Hi
It seems that the io.ktor.plugin.features.JreVersion no longer exists. So this is not compiling:
ktor {
docker {
jreVersion.set(io.ktor.plugin.features.JreVersion.JRE_17)....
Generator
new project wizard for Ktor: newly-created project fails to load with Unresolved reference: slf4j_version
Create a dummy new project using File | New | Project | Ktor
with default settings.
Wait for the project to be imported.
It won't import and fail with error
e: /Users/dmitrii.naumenko/Desktop/dev/ktor-sample/build.gradle.kts:28:42: Unresolved reference: slf4j_version
I don't see any slf4j_version
in any configuration file in the newly created project.
It's quite a poor Kotlin newcomer experience, especially for an Ultimate feature
IntelliJ IDEA 2023.2.2 (Ultimate Edition)
Build #IU-232.9921.88, built on September 30, 2023
Licensed to JetBrains Team / Dmitrii Naumenko
You have a perpetual fallback license for this version.
Subscription is active until December 28, 2026.
Runtime version: 17.0.8+7-b1000.22 aarch64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 13.5.2
GC: G1 Young Generation, G1 Old Generation
Memory: 8000M
Cores: 12
Metal Rendering is ON
Registry:
debugger.auto.attach.from.console=true
ide.experimental.ui.show.resume=true
ide.smart.update=true
ide.mac.file.chooser.native=false
ide.experimental.ui=true
ea.indicator.blinking.timeout=0
shared.indexes.jdk.download=false
scala.erase.compiler.process.jdk.once=false
shared.indexes.maven.download=false
shared.indexes.download.auto.consent=true
scala.xray.mode=true
Non-Bundled Plugins:
de.netnexus.camelcaseplugin (3.0.12)
org.nik.presentation-assistant (1.0.10)
org.jetbrains.idea.grammar (2022.3.2)
jetbrains.team.auth (232.9921.32)
com.intellij.sisyphus (232.9921.31)
com.jetbrains.intellij.code.search.polaris (0.232.900.117)
io.intellij-sdk-thread-access (2.2.1)
com.jetbrains.idea.safepush (232.9921.32)
com.intellij.ml.llm (232.9943)
com.jetbrains.intellij.api.watcher (232.9921.31)
Pythonid (232.9921.47)
org.intellij.scala (2023.2.843)
com.intellij.git.instant (1.3.7)
Kotlin: 232-1.9.0-IJ9921.88
Server
MicrometerMetricsConfig default registry leaks coroutine
MicrometerMetricsConfig always creates LoggingMeterRegistry()
by default, later if register is reset to something else, LoggingMeterRegister
won't get closed automatically.
pluginOrNull(MicrometerMetrics)
?: install(MicrometerMetrics) {
registry = AdminModule.meterRegistry
timers { call, _ -> jsonRpcMetricTags(call).forEach { (k, v) -> tag(k, v) } }
}
the coroutine created by LoggingMeterRegister
leaks. a workaround is to add registry.close
before re-assign. This is counter intuitive, most user won't do it.
There are several options,
- make the default
registery
a no-op, this may affect open box experience, but it can document it better, - use a setter for register, in setter, it should always close previous resource.
The "charset=UTF-8" part is automatically added to the `application/json` Content-Type
To reproduce run the following test:
@Test
fun test() = testApplication {
environment {
config = MapApplicationConfig()
}
routing {
get {
call.respondText("Hello", contentType = ContentType.Application.Json)
}
}
val r = client.get("/")
assertEquals("application/json", r.contentType().toString())
}
Other
NPE in JavaClientEngine body() call
There was an error when I used the AI Assistant.
- Error in UI
Unknown error. Try to repeat your request. If the issue persists, contact us.
- Error in
idea.log
INFO - STDERR - Exception in thread "DefaultDispatcher-worker-5" java.lang.NullPointerException: {
INFO - STDERR - sendAsync(http…xt)).await().body()
INFO - STDERR - } must not be null
INFO - STDERR - at io.ktor.client.engine.java.JavaHttpResponseKt.executeHttpRequest(JavaHttpResponse.kt:19)
INFO - STDERR - at io.ktor.client.engine.java.JavaHttpResponseKt$executeHttpRequest$1.invokeSuspend(JavaHttpResponse.kt)
INFO - STDERR - at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
INFO - STDERR - at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
INFO - STDERR - at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
INFO - STDERR - at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
INFO - STDERR - at java.base/java.lang.Thread.run(Thread.java:833)
INFO - STDERR - Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [CoroutineName(com.intellij.ml.llm.core.chat.services.AIAssistantServiceProjectScope), StandaloneCoroutine{Cancelling}@ff251b2, Dispatchers.Default]
- IDEA Version.
IntelliJ IDEA 2023.2 (Ultimate Edition)
Build #IU-232.8660.185, built on July 26, 2023
Subscription is active until December 14, 2023.
Runtime version: 17.0.7+7-b1000.6 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Windows 10.0
GC: G1 Young Generation, G1 Old Generation
Memory: 8192M
Cores: 12
Registry:
debugger.new.tool.window.layout=true
ide.experimental.ui=true
editor.minimap.enabled=true
Non-Bundled Plugins:
com.intellij.javafx (1.0.4)
Batch Scripts Support (1.0.13)
dev.meanmail.plugin.nginx-intellij-plugin (2022.1.1)
com.intellij.nativeDebug (232.8660.142)
org.jetbrains.idea.grammar (2022.3.2)
com.intellij.guice (232.8660.142)
com.jetbrains.jax.ws (232.8660.142)
com.intellij.plugin.adernov.powershell (2.0.10)
PlantUML integration (6.3.0-IJ2023.2)
Pythonid (232.8660.185)
org.exbin.deltahex.intellij (0.2.8.1)
ru.adelf.idea.dotenv (2023.2)
androidx.compose.plugins.idea (232.8660.142)
com.intellij.ml.llm (232.9353)
org.jetbrains.plugins.docker.gateway (232.8660.202)
Kotlin: 232-1.9.0-IJ8660.185
-
AI Assistant version:
232.9353
-
Note: In my company, I must use a proxy server with authentication to connect to the internet.
2.3.3
released 2nd August 2023
Client
Uncaught Kotlin exception: kotlin.IllegalArgumentException: Failed to open iconv for charset UTF-8 with error code 22
We're investigating the impact of iOS 17 & Xcode 15 on our software, which is partly developed using Kotlin Multiplatform. We currently use Ktor 1.6.8 in our common code.
With Xcode 15 one of our unit test fails, while it worked fine with Xcode 14, with this error:
Uncaught Kotlin exception: kotlin.IllegalArgumentException: Failed to open iconv for charset UTF-8 with error code 22
This is the Ktor related part of the stack trace:
Uncaught Kotlin exception: kotlin.IllegalArgumentException: Failed to open iconv for charset UTF-8 with error code 22
kfun:kotlin.Throwable#<init>(kotlin.String?){} + 88 (/Users/teamcity1/teamcity_work/c3a91df21e46e2c8/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Throwable.kt:24:37)
kfun:kotlin.Exception#<init>(kotlin.String?){} + 86 (/Users/teamcity1/teamcity_work/c3a91df21e46e2c8/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 86 (/Users/teamcity1/teamcity_work/c3a91df21e46e2c8/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
kfun:kotlin.IllegalArgumentException#<init>(kotlin.String?){} + 86 (/Users/teamcity1/teamcity_work/c3a91df21e46e2c8/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:59:44)
kfun:io.ktor.utils.io.charsets.checkErrors#internal + 836 (/Users/administrator/Documents/agent/work/49d4a482a8522285/ktor-io/posix/src/io/ktor/utils/io/charsets/CharsetNative.kt:54:15)
kfun:io.ktor.utils.io.charsets.CharsetImpl.<init>#internal + 782 (/Users/administrator/Documents/agent/work/49d4a482a8522285/ktor-io/posix/src/io/ktor/utils/io/charsets/CharsetNative.kt:27:9)
kfun:io.ktor.utils.io.charsets.Charsets#<init>(){} + 224 (/Users/administrator/Documents/agent/work/49d4a482a8522285/ktor-io/posix/src/io/ktor/utils/io/charsets/CharsetNative.kt:381:40)
InitSingletonStrict + 964
kfun:io.ktor.http.cio.internals.$DefaultHttpMethods$lambda-1$FUNCTION_REFERENCE$16.$<bridge-BNNNN>invoke(-1:0;-1:1){}kotlin.Char#internal + 1548
Kotlin_initRuntimeIfNeeded + 1572
+[KotlinBase allocWithZone:] + 23
Client: Target linuxArm64
I am trying to build my project with linuxArm64 as target and get the following error:
Execution failed for task ':compileKotlinLinuxArm64'.
> Could not resolve all files for configuration ':linuxArm64CompileKlibraries'.
> Could not resolve io.ktor:ktor-client-core-native:1.3.2.
Required by:
project :
> Unable to find a matching variant of io.ktor:ktor-client-core-native:1.3.2:
- Variant 'iosArm32-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'ios_arm32'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'iosArm64-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'ios_arm64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'iosX64-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'ios_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'js-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'js'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
- Variant 'js-runtime' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attributes:
- Required org.gradle.usage 'kotlin-api' and found incompatible value 'kotlin-runtime'.
- Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'js'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
- Variant 'jvm-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'jvm'.
- Other attributes:
- Found org.gradle.libraryelements 'jar' but wasn't required.
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'java-api'.
- Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
- Variant 'jvm-runtime' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'jvm'.
- Other attributes:
- Found org.gradle.libraryelements 'jar' but wasn't required.
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'java-runtime'.
- Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
- Variant 'linuxX64-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'linux_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'macosX64-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'macos_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'metadata-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'common'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
- Variant 'mingwX64-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'mingw_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'tvosArm64-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'tvos_arm64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'tvosX64-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'tvos_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'watchosArm32-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'watchos_arm32'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'watchosArm64-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'watchos_arm64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'watchosX86-api' capability io.ktor:ktor-client-core-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'watchos_x86'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
> Could not resolve io.ktor:ktor-client-curl:1.3.2.
Required by:
project :
> Unable to find a matching variant of io.ktor:ktor-client-curl:1.3.2:
- Variant 'linuxX64-api' capability io.ktor:ktor-client-curl:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'linux_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'macosX64-api' capability io.ktor:ktor-client-curl:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'macos_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'metadata-api' capability io.ktor:ktor-client-curl:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'common'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
- Variant 'mingwX64-api' capability io.ktor:ktor-client-curl:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'mingw_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
> Could not resolve io.ktor:ktor-client-logging-native:1.3.2.
Required by:
project :
> Unable to find a matching variant of io.ktor:ktor-client-logging-native:1.3.2:
- Variant 'iosArm32-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'ios_arm32'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'iosArm64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'ios_arm64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'iosX64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'ios_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'js-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'js'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
- Variant 'js-runtime' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attributes:
- Required org.gradle.usage 'kotlin-api' and found incompatible value 'kotlin-runtime'.
- Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'js'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
- Variant 'jvm-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'jvm'.
- Other attributes:
- Found org.gradle.libraryelements 'jar' but wasn't required.
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'java-api'.
- Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
- Variant 'jvm-runtime' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'jvm'.
- Other attributes:
- Found org.gradle.libraryelements 'jar' but wasn't required.
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'java-runtime'.
- Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
- Variant 'linuxX64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'linux_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'macosX64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'macos_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'metadata-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.platform.type 'native' and found incompatible value 'common'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.native.target 'linux_arm64' but no value provided.
- Variant 'mingwX64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'mingw_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'tvosArm64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'tvos_arm64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'tvosX64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'tvos_x64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'watchosArm32-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'watchos_arm32'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'watchosArm64-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'watchos_arm64'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
- Variant 'watchosX86-api' capability io.ktor:ktor-client-logging-native:1.3.2:
- Incompatible attribute:
- Required org.jetbrains.kotlin.native.target 'linux_arm64' and found incompatible value 'watchos_x86'.
- Other attributes:
- Found org.gradle.status 'release' but wasn't required.
- Required org.gradle.usage 'kotlin-api' and found compatible value 'kotlin-api'.
- Required org.jetbrains.kotlin.platform.type 'native' and found compatible value 'native'.
And linuxX64 works fine in the same project.
s-maxage is not used, even if `HttpCache.Config.isShared` is true
When HttpCache.Config.isShared
is true, should be give priority to s-maxage over max-age.
This is why should be give priority to s-maxage over max-age.
https://httpwg.org/specs/rfc7234.html#rfc.section.5.2.2.9
But s-maxage is not used because isShared
member is not passed for a parameter of cache storing function.
Cache returns null when vary header set different ways whatever it has same values
DefaultRequest: a cookie appears twice in the request header when sending a request with another cookie
Hey hey, first time creating an issue. Hope I'm doing everything correctly :-)
I'm facing this problem, when I try to prepare a HttpClient
like this:
val httpClient = HttpClient(CIO) {
engine {
endpoint.connectTimeout = 10000
}
defaultRequest {
url(defaultUrl)
cookie("first-cookie", "foo")
}
}
val request = httpClient.preparePost("/some-route") {
headers {
contentType(ContentType.Application.Json)
}
cookie("second-cookie", "bar")
}
request.execute()
This will send a request with the following cookies in the header:
INFO io.ktor.client.HttpClient - REQUEST: https://domain/some-route
METHOD: HttpMethod(value=POST)
COMMON HEADERS
-> Accept: application/json
-> Accept-Charset: UTF-8
-> Cookie: first-cookie=foo; first-cookie=foo; second-cookie=bar
CONTENT HEADERS
-> Content-Length: 34
-> Content-Type: application/json
BODY Content-Type: application/json
BODY START
... some body ...
BODY END
Any ideas why this happens?
Doing it like this instead would work (removing the first cookie from the defaultRequest
:
val httpClient = HttpClient(CIO) {
engine {
endpoint.connectTimeout = 10000
}
defaultRequest {
url(defaultUrl)
}
}
val request = httpClient.preparePost("/some-route") {
headers {
contentType(ContentType.Application.Json)
}
cookie("first-cookie", "foo")
cookie("second-cookie", "bar")
}
request.execute()
Thanks for your help already!
Server
KtorServlet does not support yaml configuration
I am using ktor on top of the Spring Boot. I am using io.ktor:ktor-server-servlet
However, it does not support yaml configuratuin. Ktor has ktor-server-config-yaml
dependency, but it does not work with Ktor Servlet.
After some investigation, I found that ktor does not support yaml configuration.
It will be nice if somebody implement this feature.
I want to configure ktor and spring boot in same application.yaml
staticFiles responds twice if both index and defaultPath are set
Given for example
staticFiles("/", File(pathToStaticFiles), index = "index.html") {
default("index.html")
}
or
singlePageApplication {
filesPath = pathToStaticFiles
}
and a request for a directory, for example GET /
, the plugin attempts to respond twice, first with the index, then the default. It seems to behave normally apart from the double 200 OK: GET - /
in the logs, but in my case it constantly causes UnsupportedOperationException: Headers can no longer be set because response was already completed for call
when another plugin tries to append headers during the second respond call.
My guess is that there's a missing if(isHandled) return
around here: https://github.com/ktorio/ktor/blob/46ac5de2495c399453e11e3078404bedcdad72ba/ktor-server/ktor-server-core/jvm/src/io/ktor/server/http/content/StaticContent.kt#L464
Server: Target linuxArm64
Analogous to https://youtrack.jetbrains.com/issue/KTOR-872, it would be very useful to have linuxArm64
support on ktor server.
Test Infrastructure
"Test engine is already completed" error while establishing Websockets connection
While reading testApplication
, I don't understand why block()
is executed before testApplication.start()
?
@KtorDsl
public fun testApplication(
parentCoroutineContext: CoroutineContext = EmptyCoroutineContext,
block: suspend ApplicationTestBuilder.() -> Unit
) {
val builder = ApplicationTestBuilder()
.apply {
runBlocking(parentCoroutineContext) {
if (parentCoroutineContext != EmptyCoroutineContext) {
environment {
this.parentCoroutineContext = parentCoroutineContext
}
}
block()
}
}
val testApplication = TestApplication(builder)
testApplication.start()
testApplication.stop()
}
Other
Drop linuxArm64 publication from ktor-client-curl
Now it's impossible to provide dependency
DelegatingTestingClientEngine fails when ContentNegotiation with protobuf is installed and empty body
Protobuf does not support empty lists or nullable values. As such if the body of the response from the server is a object like:
data class MyClass(val aList: List<String> = emptyList())
and the list is empty, the body of the response will be empty.
This implies that when deserializing:
// io.ktor:ktor-serialization:2.3.2
package io.ktor.serialization
// ...
public suspend fun List<ContentConverter>.deserialize(
body: ByteReadChannel,
typeInfo: TypeInfo,
charset: Charset
): Any {
val result = asFlow()
.map { converter -> converter.deserialize(charset = charset, typeInfo = typeInfo, content = body) }
.firstOrNull { it != null || body.isClosedForRead }
return when {
result != null -> result
!body.isClosedForRead -> body
typeInfo.kotlinType?.isMarkedNullable == true -> NullBody
else -> throw ContentConvertException("No suitable converter found for $typeInfo")
}
}
The channel is always closed for read, making result
a null
and always going for the the else
branch of the when
.
I am not sure if it happens only to the test engine provided by the test server, but given the code running above, I assume not.
java.util.zip.DataFormatException after enabling permessage-deflate
We have enabled WebSocket compression by default:
install(WebSockets) {
extensions {
install(WebSocketDeflateExtension)
}
}
And now we're getting the following exception:
java.util.zip.DataFormatException invalid distance too far back
at java.base/java.util.zip.Inflater.$$YJP$$inflateBytesBytes(Native Method)
at java.base/java.util.zip.Inflater.inflateBytesBytes(Inflater.java)
at java.base/java.util.zip.Inflater.inflate(Inflater.java:378)
at io.ktor.websocket.internals.DeflaterUtilsKt.inflateFully(DeflaterUtils.kt:50)
at io.ktor.websocket.WebSocketDeflateExtension.processIncomingFrame(WebSocketDeflateExtension.kt:140)
at io.ktor.websocket.DefaultWebSocketSessionImpl.processIncomingExtensions(DefaultWebSocketSession.kt:325)
at io.ktor.websocket.DefaultWebSocketSessionImpl.access$processIncomingExtensions(DefaultWebSocketSession.kt:70)
at io.ktor.websocket.DefaultWebSocketSessionImpl$runIncomingProcessor$1.invokeSuspend(DefaultWebSocketSession.kt:204)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:233)
at kotlinx.coroutines.DispatchedTaskKt.resumeUnconfined(DispatchedTask.kt:189)
at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:161)
at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:474)
at kotlinx.coroutines.CancellableContinuationImpl.completeResume(CancellableContinuationImpl.kt:590)
at kotlinx.coroutines.channels.BufferedChannelKt.tryResume0(BufferedChannel.kt:2896)
at kotlinx.coroutines.channels.BufferedChannelKt.access$tryResume0(BufferedChannel.kt:1)
at kotlinx.coroutines.channels.BufferedChannel$BufferedChannelIterator.tryResumeHasNext(BufferedChannel.kt:1689)
at kotlinx.coroutines.channels.BufferedChannel.tryResumeReceiver(BufferedChannel.kt:642)
at kotlinx.coroutines.channels.BufferedChannel.updateCellSend(BufferedChannel.kt:458)
at kotlinx.coroutines.channels.BufferedChannel.access$updateCellSend(BufferedChannel.kt:36)
at kotlinx.coroutines.channels.BufferedChannel.send$suspendImpl(BufferedChannel.kt:3089)
at kotlinx.coroutines.channels.BufferedChannel.send(BufferedChannel.kt)
at io.ktor.websocket.RawWebSocketJvm$1.invokeSuspend(RawWebSocketJvm.kt:68)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:833)
CIO ConnectionFactory leaks on cancellation
Not compatible with kotlinx-html 0.9.1
kotlinx-html has dropped the onTagError
method (https://github.com/Kotlin/kotlinx.html/pull/224). The ktor integration currently relies on this. When updating the html dependency, errors cannot be handled properly anymore:
java.lang.NoSuchMethodError: 'void kotlinx.html.TagConsumer.onTagError(kotlinx.html.Tag, java.lang.Throwable)'
at io.ktor.server.html.RespondHtmlKt.respondHtml(RespondHtml.kt:68)
at io.ktor.server.html.RespondHtmlTemplateKt.respondHtmlTemplate(RespondHtmlTemplate.kt:21)
at io.ktor.server.html.RespondHtmlTemplateKt.respondHtmlTemplate$default(RespondHtmlTemplate.kt:15)
at
Add system property to disable automatic installation of runtime shutdown hook
2.3.2
released 29th June 2023
Build System Plugin
Compatibility with Gradle 8.0 release
Ktor sets main class using deprecated method because of dependency on shadow plugin v6, that results in following warning:
The JavaApplication.setMainClassName(String) method has been deprecated. This is scheduled to be removed in Gradle 8.0. Use #getMainClass().set(...) instead. See https://docs.gradle.org/7.6/dsl/org.gradle.api.plugins.JavaApplication.html#org.gradle.api.plugins.JavaApplication:mainClass for more details.
Shadow plugin v7 already released, so it can be updated in Ktor plugin if there are no minimal gradle version requirements.
Client
ContentType of a response body isn't set inside OkHttp's interceptor when a form request is sent
I am in the process of migrating from retrofit to ktor. I am using OkHttp to keep the interceptor logic. One of these interceptors requires that the "contentType" field is not null for logging.
I use the FormDataContent class to pass parameters to the form according to https://ktor.io/docs/request.html#form_parameters:
// class io.ktor.client.request.forms.FormDataContent
public class FormDataContent(
public val formData: Parameters
) : OutgoingContent.ByteArrayContent() {
private val content = formData.formUrlEncode().toByteArray()
override val contentLength: Long = content.size.toLong()
override val contentType: ContentType = ContentType.Application.FormUrlEncoded.withCharset(Charsets.UTF_8) <----- setting form content type
override fun bytes(): ByteArray = content
}
But when converting to OkHtpp classes, the "contentType" parameter is lost:
// class io.ktor.client.engine.okhttp.OkHttpEngine
@OptIn(DelicateCoroutinesApi::class)
internal fun OutgoingContent.convertToOkHttpBody(callContext: CoroutineContext): RequestBody = when (this) {
is OutgoingContent.ByteArrayContent -> bytes().let {
it.toRequestBody(null, 0, it.size) <----- why null?
}
is OutgoingContent.ReadChannelContent -> StreamRequestBody(contentLength) { readFrom() }
is OutgoingContent.WriteChannelContent -> {
StreamRequestBody(contentLength) { GlobalScope.writer(callContext) { writeTo(channel) }.channel }
}
is OutgoingContent.NoContent -> ByteArray(0).toRequestBody(null, 0, 0)
else -> throw UnsupportedContentTypeException(this)
}
please tell me is this behavior is correct? how can i fix it?
Core
Linking release build leads to compilation error with coroutines of version 1.7.0-Beta
On Kotlin 1.8.10 and 1.8.20-RC2, the following error occurs during :linkReleaseExecutableMingwX64
Compilation failed: Internal compiler error: no implementation found for FUN DEFAULT_PROPERTY_ACCESSOR name:<get-parent> visibility:public modality:ABSTRACT <> ($this:kotlinx.coroutines.Job) returnType:kotlinx.coroutines.Job?
when building itable for CLASS INTERFACE name:Job modality:ABSTRACT visibility:public superTypes:[kotlin.coroutines.CoroutineContext.Element]
implementation in CLASS CLASS name:ChannelJob modality:FINAL visibility:private superTypes:[io.ktor.utils.io.ReaderJob; io.ktor.utils.io.WriterJob; kotlinx.coroutines.Job]
at Z:/buildAgent/work/8d547b974a7be21f/ktor-io/common/src/io/ktor/utils/io/Coroutines.kt (156:1)
CLASS CLASS name:ChannelJob modality:FINAL visibility:private superTypes:[io.ktor.utils.io.ReaderJob; io.ktor.utils.io.WriterJob; kotlinx.coroutines.Job]
* Source files:
* Compiler version info: Konan: 1.8.20-RC2 / Kotlin: 1.8.20
* Output kind: PROGRAM
But, linkDebugExecutableMingwX32 works.
I only tried on Linux
Docs
Fix OAuth tutorial to mention Sessions plugin
https://ktor.io/docs/oauth.html is using sessions without installing the plugin. It will lead to the runtime error
Generator
Wrong comment for reference to main function in project with configuration in application.yaml
Create new project with configuration in YAML
In Application.kt file there is a comment about application.conf references for the main function, but configuration is in application.yaml file.
Server
MapApplicationConfig removes deeply nested properties when converting to a map
val app: TestApplication = TestApplication {
environment {
val twiceNestedValue = MapApplicationConfig().apply { put("ktor.database.jdbcURL", TestDbContainer.getDbURL()) }
val onceNestedValue = MapApplicationConfig().apply { put("database.jdbcURL", TestDbContainer.getDbURL()) }
log.info("twiceNestedValue: ${twiceNestedValue.toMap()}")
log.info("onceNestedValue: ${onceNestedValue.toMap()}")
}
}
Output:
INFO ktor.test - twiceNestedValue: {ktor={database={}}}
INFO ktor.test - onceNestedValue: {database={jdbcURL=jdbc:postgresql://localhost:49250/somedb?loggerLevel=OFF}}
Creating config at runtime in this manner is the JetBrains suggested solution https://stackoverflow.com/a/75590178/9397815
The error message is not helpful when authenticating with a bearer header with a colon
Let's consider this request:
METHOD: HttpMethod(value=POST)
COMMON HEADERS
-> Accept: application/json
-> Accept-Charset: UTF-8
-> Authorization: Bearer fake:0
CONTENT HEADERS
-> Content-Length: 27
-> Content-Type: application/json
BODY Content-Type: application/json
BODY START
{"name":"A new department"}
BODY END
Using Ktor 2.2.1, using the Authentication
plugin with bearer {}
, receiving this request instantly returns a 400 Bad Request
with no body. Because I used the ContentNegotiation
plugin as well, the error message I got was
Unknown: 400 Bad Request with no provided body
with a previously logged line
TRACE i.k.s.p.c.ContentNegotiation - Skipping because the type is ignored.
leading me in a completely wrong direction.
I'm not sure having a colon in a Bearer token is even allowed by the RFCs, but if it isn't the error message should be clearer that this is the issue. For example, returning an error message, or logging the clear explanation.
So far the only actually relevant log I could find was:
TRACE i.k.s.auth.Authentication - Trying to authenticate /test with null
which doesn't help.
Other
Cache returns null when vary header has more fields in the cached response
Update Kotlin to 1.8.22
2.3.1
released 1st June 2023
Client
Bearer auth token refresh hangs after prior refresh threw an exception
When BearerAuthProvider
calls AuthTokenHolder.setToken
, it passes a block that is configurable by ktor consumers (unliked BasicAuthProvider
and DigestAuthProvider
). When this block throws an exception, it results in the skipping of newDeferred.complete(newTokens)
and refreshTokensDeferred.value = null
, which causes the next call to setToken
to hang indefinitely on deferred.await()
. Here are the relevant blocks of code:
A workaround for this is for ktor consumers to wrap their refreshTokens
logic in a try block that catches Throwable
like this:
install(Auth) {
bearer {
loadTokens {
// ...
}
refreshTokens {
try {
// ...
} catch (t: Throwable) {
// don't allow this to bubble up
}
}
}
}
Ktor Client Unable to Stream Responses in Javascript
Summary
Ktor 2.3.0 client under Javascript is unable to stream responses from a server. The same code works correctly with Ktor running under the JVM.
Steps to reproduce
- Configure a server to emit responses as a stream
- Configure a ktor client in Javascript to read the response as a stream (e.g.
response.bodyAsChannel()
and reading response line by line)
With the attached sample application
- Run the server ./gradlew :server-app:run and make sure you have a console to look at the server log output
- Launch your browser and open the developer tools
- Visit http://localhost:8080
- Compare the logs from the client and server
Results
Expected
Javascript client reads stream as each item is written
Actual
Javascript buffers until entire response is complete before actually reading it.
Example output from the test application
Server log output:
Writing H at 2023-04-29T16:56:47.240733Z
Writing e at 2023-04-29T16:56:47.750986Z
Writing l at 2023-04-29T16:56:48.256360Z
Writing l at 2023-04-29T16:56:48.761753Z
Writing o at 2023-04-29T16:56:49.264868Z
Writing at 2023-04-29T16:56:49.765610Z
Writing w at 2023-04-29T16:56:50.270883Z
Writing o at 2023-04-29T16:56:50.776197Z
Writing r at 2023-04-29T16:56:51.283105Z
Writing l at 2023-04-29T16:56:51.788978Z
Writing d at 2023-04-29T16:56:52.292238Z
Browser log output:
Received H at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received e at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received l at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received l at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received o at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received w at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received o at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received r at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received l at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Received d at Sat Apr 29 2023 12:56:52 GMT-0400 (Eastern Daylight Time)
Notes
I reproduced this with Ktor 2.3.0. I have not tested older versions to see whether this is a long standing bug or a regression.
AndroidClientEngine: the engine double-parses query parameters before sending a request
We have an API that generates a document served by AWS cloudfront and returns a URL to download the document when it is complete. The problem is that the URL has an extra leading & in the query as in: "?&Expires". When the Ktor client makes the call it drops the leading ampersand which in my mind is correct since it really has no meaning. But unfortunately without the ampersand the call fails.
There is no way to force Ktor not to drop the ampersand.
Cloudfront should not be generating a URL like this and even worse should not be treating the presence as significant and I have a back end person looking into it, but as it stands now we have to have the extra ampersand and will have to hack and expect/actual API to use the platform specific Http Client.
Electron/Node.js detection doesn't work correctly
After testing the KTOR-5650 fix (https://github.com/ktorio/ktor/pull/3456) in Electron the check turned out to return false even though we have the Node API. In Electron at this point there can be two process
objects and the correct one is defined on window.process
, not just process
. So we'd have to first detect which process
object to use and after that the remaining .versions.node
check can be done.
A coroutine closed due to cancellation is considered by the JsWebSocketSession to be closed on failure
Expected behavior: Coroutine cancellation leads WebSocket to be closed by Normal Closure (1000) reason or like so
Actual behavior: Any coroutine completion with non-null cause leads WebSocket to be closed by Internal Error (1011) reason
Also, this reason for closing is not supported by browsers, which causes an invalid access error, leaving the WebSocket open.
Server
Multipart: Support not writing a temporary file for binary data
I am attempting to migrate an existing netty based application to ktor.
One of its features is handling large file uploads (100mb -> 10Gb).
We are handle sensitive personal information and would prefer not write anything to disk.
I can't see a way to implement this using the existing multipart file upload support in Ktor.
Is there a way to directly access a ReadByteChannel for the file and avoid the tmp disk write?
YAML config does not support reading variables from itself
For example, in HOCON I can write the following:
value {
my = "My value"
}
config {
database {
value = ${value.my}
}
}
If I write similar config file in YAML:
value:
my: "My value"
config:
database:
value: $value.my
It will be failing to read $value.my
assuming it's an environment variable
HOCON: "No configuration setting found for key" error after merging
Steps to reproduce:
- Create a Ktor application
import io.ktor.server.application.*
import io.ktor.server.cio.*
import io.ktor.server.routing.*
fun main(args: Array<String>) {
EngineMain.main(args)
}
@Suppress("unused")
fun Application.main() {
routing {
}
}
- Create two external config files
application.conf
ktor {
development = false
deployment {
host = "0.0.0.0"
port = 8080
shutdownGracePeriod = 2000
shutdownTimeout = 60000
}
application {
modules = [package.MainKt.main]
}
}
extra.conf
// Ktor can't load additional configuration without "ktor" definition
//ktor {
// deployment {
// }
//}
app {
test = "conf"
}
- Run the app using external configuration files
-config=application.conf -config=extra.conf
The result
Actual
It fails with Exception in thread "main" com.typesafe.config.ConfigException$Missing: extra.conf: 1: No configuration setting found for key 'ktor'
Expected
The app should run since the ktor configuration is defined in the application.conf file
As a workaround, it works when empty ktor and deployment configurations are added to the extra.conf file.
The Logging plugin doesn't log full kotlinx deserialization errors
KotlinxSerializationConverter#deserialize()
“hides” actual deserialization error messages:
// KotlinxSerializationConverter.kt
fun deserialize(): Any? {
try {
// Deserialize and return if successful.
} catch (cause: Throwable) {
throw JsonConvertException("Illegal input", cause)
}
}
While the message will be available in the cause
of the thrown JsonConvertException
, it causes the Logging
plugin to display some pretty useless logs:
// Logging.kt
fun logResponseException(log: StringBuilder, request: HttpRequest, cause: Throwable) {
log.append("RESPONSE ${request.url} failed with exception: $cause")
}
The result is logs like this, which unfortunately doesn't provide much value:
RESPONSE https://host/path failed with exception: io.ktor.serialization.JsonConvertException: Illegal input
Is there a way we can provide cause.message
alongside the "Illegal input"
message from KotlinxSerializationConverter
? Perhaps just something like this:
throw JsonConvertException("Illegal input: ${cause.message}", cause)
XForwardedHeaders should set `remoteAddress` in addition to `remoteHost`
Currently, upon receipt of an `X-Forwarded-For` header, the XForwardedHeaders plugin sets the remoteHost
origin property:
If the value extracted from the header is an address rather than a host, Ktor should be setting the remoteAddress
property.
The KDocs for these two properties are:
/** Client address or host name if it can be resolved. ... */
public val remoteHost: String
and
/** Client address. ... */
public val remoteAddress: String
which clearly implies that for an address, these two values should be the same. The kdocs for remoteAddress
even specify that it can be overwridden via the forwarded headers.
Therefore, client code that wants the remote address should clearly be using remoteAddress
, and yet when the XForwardedHeaders plugin is used and the headers resolve to an address, the remoteHost
will contain the resolved remote address, but remoteAddress
will not.
Sessions: Set-Cookie is added on every api request.
Hi, I'm new to Ktor.
please understand if my question is wrong.
when I use session like below
install(Sessions) {
cookie<User>(
USER_COOKIE_NAME,
directorySessionStorage(File(".sessions"), cached = true)
)
I set session on signin
api is called.
and client saved the sessionId and delivered the sessionId to server on getUser
api like below
and server response with Set-Cookie header even getUser
api controller didn't set session.
REQUEST: http://192.168.224.95:8080/getUser
METHOD: HttpMethod(value=POST)
COMMON HEADERS
-> KEY: Header test
-> Accept: application/json
-> Accept-Charset: UTF-8
-> Cookie: user=4879fe5aee4c5b3d;
CONTENT HEADERS
BODY Content-Type: application/json
BODY Content-Type: application/json; charset=UTF-8
BODY START
RESPONSE: 200 OK
METHOD: HttpMethod(value=POST)
FROM: http://192.168.224.95:8080/getUser
COMMON HEADERS
-> Connection: keep-alive
-> Content-Length: 21
-> Content-Type: application/json; charset=UTF-8
-> Set-Cookie: simple-user=4879fe5aee4c5b3d; Max-Age=604800; Expires=Thu, 20 Aug 2020 15:57:08 GMT; Path=/; HttpOnly; $x-enc=URI_ENCODING
-> X-Android-Received-Millis: 1597334141402
-> X-Android-Response-Source: NETWORK 200
-> X-Android-Selected-Protocol: http/1.1
-> X-Android-Sent-Millis: 1597334137063
{"id":"a","name":"b"}
BODY END
so, I checked which part add Set-Session
and found this code on io.ktor.sessions.Sessions
pipeline.intercept(ApplicationCallPipeline.Features) {
val providerData = sessions.providers.associateBy({ it.name }) {
it.receiveSessionData(call)
}
val sessionData = SessionData(sessions, providerData)
call.attributes.put(SessionKey, sessionData)
}
in my past experience, Set-Session
is added only when session is set.
But, my experience is not long, so, kindly understand if I'm wrong.
I would like to ask if this is intended behavior.
Thanks to read.
Please let me know if further information is required.
Significant delay between getting a part and starting reading from its provider for multipart/form-data requests
I have Ktor server in an Android app and I use it to receive files. I would like to display a progress bar in my app for the duration it will take for the receiving file to arrive.
Some sort of a listener with bytesSent/bytesTotal would be perfect
Just to clarify, I am not interested in listening to upload progress from a client, but the time it takes to receive a files from the server.
Right now my incoming files handling code looks like this:
// 1
call.receiveMultipart().forEachPart { part ->
// 2
when (part) {
is PartData.FileItem -> {
val input = part.provider()
// Read from input by chunks
}
else -> {}
}
}
I would like to listen to the progress between (1) and (2). Right now (1) is called, then the file is fully uploaded to the server, and then (2) is called.
WebSockets: requests to a non-existing route cause server to lock up after responding with 404 (potential DOS)
Platform: JVM
We recently observed our production application servers locking up and becoming unresponsive due to legacy clients making a large number of web socket upgrade requests to a URI for which a previous route
no longer existed. This is equivalent to a denial-of-service attack. We traced the issue back to the simple reproduction below, which also exposes a Ktor web socket client ambiguity on how to handle 404 and other connection issues: the documentation is not clear about what happens in those cases. If catching an exception is the way to go, as in the reproduction below, please update the documentation.
The issue appears to be that the server pipeline for web socket upgrade requests never completes if the client's upgrade request is sent to a URI for which no web socket handler route exists. The attached reproduction starts a server and then attempts to make 1,000 web socket connections. As is, this succeeds and at the end of the test run, roughly 10 coroutines exist on the server.
If the configured route is changed from /test/{id}
to /test
, the server locks up after processing ~200 requests, meaning it doesn't become unresponsive immediately, but only past a certain load. Over 4,000 suspended coroutines exist on the server at the end of the failed test run.
The workaround we're testing right now is to accept and immediately drop the connections to the known legacy route, but that doesn't resolve the issue that a potential attacker who knows that Ktor with web socket support is running on our servers could craft a large number of web socket requests to a different non-existing route and cause application servers to become unresponsive!
Support optional properties in YAML
In HOCON there are 3 scenarios of variable declaration:
a) With default value
myValue = "default value"
myValue = ${?MY_ENV_VAR}
myValue
equals "default value" if one doesn't provide MY_ENV_VAR
b) Without default value and with question mark
myValue = ${?MY_ENV_VAR}
myValue
equals the value of MY_ENV_VAR
or does not exist otherwise (.getProperty
returns null
)
c) Without default value and with no question mark
myValue = $MY_ENV_VAR
myValue
equals the value of MY_ENV_VAR
which must be present (.getProperty
throws an exception in attempt to read MY_ENV_VAR
)
There are only 2 equivalents in YAML:
a) With default value
myValue = "$MY_ENV_VAR:default value"
myValue
equals "default value" if one doesn't provide MY_ENV_VAR
c) With no default value
myValue = $MY_ENV_VAR
myValue
equals the value of MY_ENV_VAR
which must be present (.getProperty
throws an exception in attempt to read MY_ENV_VAR
)
The only way to achieve "b" in YAML is to write ugly code like the following introducing an extra "NULL" string (Or any other token that one believes will never present in real values):
my:
config: "$MY_ENV_VAR:NULL"
And then handling it like the following.
fun ApplicationConfig.yamlStringPropertyOrNull(path: String): String? = propertyOrNull(path)
?.getString()
?.takeIf { it != "NULL" }
fun ApplicationConfig.yamlListProperty(path: String): List<String> = property(path)
.getList()
.filter { it != "NULL" }
val myValue get(): MyConfig = conf.yamlStringPropertyOrNull("my.config")
Test Infrastructure
testApplication: NPE when test server doesn't reply with an HTTP upgrade
When using the example here I get a java.lang.NullPointerException
when running the test. Am I doing something wrong? My hope is to be able to test my WebSocket client of my KMP-app and I got the impression that this might be able to help me?
Using Ktor 2.2.4 and Kotlin 1.8.10.
Here's the stacktrace of the crash. Thanks in advance.
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
java.lang.NullPointerException
at io.ktor.server.testing.TestApplicationEngineJvmKt$handleWebSocketConversationNonBlocking$5.invokeSuspend(TestApplicationEngineJvm.kt:73)
(Coroutine boundary)
at io.ktor.server.testing.TestApplicationEngineJvmKt.handleWebSocketConversationNonBlocking(TestApplicationEngineJvm.kt:68)
at io.ktor.server.testing.client.TestHttpClientEngineBridge.runWebSocketRequest(TestHttpClientEngineBridgeJvm.kt:34)
at io.ktor.server.testing.client.TestHttpClientEngine.execute(TestHttpClientEngine.kt:50)
at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:99)
at io.ktor.client.engine.HttpClientEngine$DefaultImpls.executeWithinCallContext(HttpClientEngine.kt:100)
at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:70)
at io.ktor.client.plugins.HttpSend$DefaultSender.execute(HttpSend.kt:138)
at io.ktor.client.plugins.HttpRedirect$Plugin$install$1.invokeSuspend(HttpRedirect.kt:64)
at io.ktor.client.plugins.HttpCallValidator$Companion$install$3.invokeSuspend(HttpCallValidator.kt:151)
at io.ktor.client.plugins.HttpSend$Plugin$install$1.invokeSuspend(HttpSend.kt:104)
at io.ktor.client.plugins.websocket.WebSockets$Plugin$install$1.invokeSuspend(WebSockets.kt:169)
at io.ktor.client.plugins.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:130)
at io.ktor.client.plugins.HttpRequestLifecycle$Plugin$install$1.invokeSuspend(HttpRequestLifecycle.kt:38)
at io.ktor.client.HttpClient.execute$ktor_client_core(HttpClient.kt:191)
at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:108)
at io.ktor.client.plugins.websocket.BuildersKt.webSocket(builders.kt:241)
at ModuleTest$testConversation$1.invokeSuspend(ModuleTest.kt:15)
at io.ktor.server.testing.TestApplicationKt$testApplication$builder$1$1.invokeSuspend(TestApplication.kt:288)
Caused by: java.lang.NullPointerException
at io.ktor.server.testing.TestApplicationEngineJvmKt$handleWebSocketConversationNonBlocking$5.invokeSuspend(TestApplicationEngineJvm.kt:73)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Other
getTimeMillis has seconds precision on native
Flaky tests in WinHttp engine
Warn when the RateLimit plugin installed after the routing
The Rate limit provider with name RateLimitName (name=KTOR_NO_NAME_RATE_LIMITER) is not configured
or a similar error doesn't explain that the RateLimit
plugin must be installed if it wasn't and that its installation must go before the one of the routing.
Curl sometimes fails with `API function called from within callback`
Darwin engine does not support streaming of request body
Allow access to RateLimiters related to call
RateLimitters for every request key live in memory forever
In case the plugin is set up to have separate limiter per user, this can cause unnecessary memory usage.
Make System Property to Set outgoingToBeProcessed Size for WebSockets
GMTDate timestamp doesn't reflect timezone when created using `Calendar.toDate` method
val calendar = Calendar.getInstance(TimeZone.getTimeZone("Europe/Berlin"))
val timestamp = calendar.toDate(null).timestamp // Equals to GMTDate().timestamp
After creating a new GMTDate
instance from this timestamp, a different date is created.
2.3.0
released 19th April 2023
Client
Ktor JS websocket client unconfigurable logging
File Websockets.kt contains own LOGGER
internal val LOGGER = KtorSimpleLogger("io.ktor.client.plugins.websocket.WebSockets")
Which can't be configured.
Implementation of KtorSimpleLoggerJs.kt
@Suppress("FunctionName")
public actual fun KtorSimpleLogger(name: String): Logger = object : Logger {
override fun error(message: String) {
console.error(message)
}
override fun error(message: String, cause: Throwable) {
console.error("$message, cause: $cause")
}
override fun warn(message: String) {
console.warn(message)
}
override fun warn(message: String, cause: Throwable) {
console.warn("$message, cause: $cause")
}
override fun info(message: String) {
console.info(message)
}
override fun info(message: String, cause: Throwable) {
console.info("$message, cause: $cause")
}
override fun debug(message: String) {
console.info("DEBUG: $message")
}
override fun debug(message: String, cause: Throwable) {
console.info("DEBUG: $message, cause: $cause")
}
override fun trace(message: String) {
console.info("TRACE: $message")
}
override fun trace(message: String, cause: Throwable) {
console.info("TRACE: $message, cause: $cause")
}
}
It floods logs with unnecessary and useless messages (why not console.trace
?):
TRACE: Sending WebSocket request [object Object]
This logger should be configurable with Logging plugin.
Prefer Node instead of browser behavior
In the engine there are several places which check first if (PlatformUtils.IS_BROWSER)
but it would be better if those checked for if (PlatformUtils.IS_NODE)
first (i.e. flip the if and else blocks) because for Electron IS_NODE and IS_BROWSER can both be true at the same time, but the browser fetch API is very crippled in comparison to the node-fetch based solution. For example you can't read response headers without explicitly setting up CORS headers in the backend.
In general, instead of having two Booleans (which can both be true) it would be better to have an enum and a single place in the code where the decision of prioritizing Node vs. browser behavior happens. Otherwise such "bugs" of prioritizing crippled browser APIs will happen all over the place and harm Electron integration.
Java engine: Websockets client sends two PONG frames for each PING frame from a server
When the server sends a PING frame, the Ktor client replies with 2 PONG frames.
Maybe the Java web socket client implementation already deals with PING/PONG automatically, and Ktor adds another layer of PING handling, resulting in 2 PONGs, but I haven't checked the code.
It is technically valid to send unsolicited PONG frames as per the web socket specification, but this is not quite what I would expect, and it's breaking Autobahn TestSuite, marking Ktor as failing ping/pong tests.
DarwinClientEngine: a request deadlocks on macOS since 2.2.2
I'm experiencing that the DarwinClientEngine deadlocks on any http requests since 2.2.2 if they are started from the main thread. If I downgrade to 2.2.1 everything works fine.
deadlocks:
HttpClient(
Darwin.create {
pipelining = true
}
)
println("making request..")
val result = httpClient().get("https://www.google.com")
println("status: $result.status")
works:
HttpClient(
Darwin.create {
pipelining = true
}
)
launch(Dispatchers.Default) {
println("making request..")
val result = httpClient().get("https://www.google.com")
println("status: $result.status")
}
BearerAuthProvider: Token is being refreshed multiple times when queued call finishes with 401 after refresh token succeeds
We are using BearerAuthProvider and experiencing unexpected consecutive refresh token calls in this particular scenario.
Here the sequence of the calls with url, response code, response time and token. The calls are in response order.
- /url1 -> 401 - response time: 310 ms - uses token1
- /url2 -> 401 - response time: 276 ms - uses token1
- /url3 -> 401 - response time: 322 ms - uses token1
- /refresh -> 200 - response time: 390 ms - refresh token 1 to token 2
- /url1 -> 200 - response time: 168 ms - uses token 2
- /url2 -> 200 - response time: 179 ms - uses token 2
- /url3 -> 200 - response time: 189 ms - uses token 2
- /url4 -> 401 - response time: 264 ms - uses token 1
- /refresh -> 200 - response time: 168 ms - refresh token 2 to token 3
...
What we see is that:
- url1, url2, url3 are queued for retry and retried after step 4 in which the refresh token is done. These retries succeed.
- url4 is started when the refresh token is in process but the 401 is received after the refresh token finishes. The 401 failure of url4 leads to another refresh token. We were expecting that this call was going to be retried immediately without a refresh token, just using token2 (refreshed token received at step 4).
The situation gets even worse if we amplify this behaviour to a higher number of calls: e.g. this could lead to url4 failing twice with 401 and this call will be lost and never complete.
DigestAuthProvider: realm sent from the server doesn't participate in the computation of `response`
realm should be set earlier in the function
should be moved to before line 154. This fixes the issues for me.
Missing cause for WebSocketException in JavaHttpWebSocket
Currently in JavaHttpWebSocket.onError
, Ktor throws a WebSocketException
with just the message from the cause:
override fun onError(webSocket: WebSocket, error: Throwable) {
val cause = WebSocketException("${error.message}")
_incoming.close(cause)
_outgoing.cancel()
socketJob.complete()
}
This is not a nice solution in general, because exception messages don't completely inform about the problem. It's the combination of exception type + message that gives the full insight. For instance, with FileNotFoundException: /tmp/foo
, we don't want to just bubble up the message /tmp/foo
, it doesn't really explain the problem.
As a more realistic example, I'm getting a "null"
message from Ktor because some exception in the Java web socket has no message, which makes this stacktrace pretty pointless:
Caused by: org.hildan.krossbow.websocket.WebSocketException: error in Ktor's websocket: null
at org.hildan.krossbow.websocket.ktor.KtorWebSocketConnectionAdapter$incomingFrames$3.invokeSuspend(KtorWebSocketClient.kt:60)
•••
at org.hildan.krossbow.websocket.test.TestUtilsJvmKt.runSuspendingTest(TestUtilsJvm.kt:8)
at org.hildan.krossbow.websocket.test.TestUtilsJvmKt.runSuspendingTest$default(TestUtilsJvm.kt:8)
at org.hildan.krossbow.websocket.test.autobahn.AutobahnClientTestSuite.runAutobahnTestCase(AutobahnClientTestSuite.kt:200)
at org.hildan.krossbow.websocket.test.autobahn.AutobahnClientTestSuite.autobahn_1_1_7_echo_text_payload(AutobahnClientTestSuite.kt:54)
•••
Caused by: io.ktor.client.plugins.websocket.WebSocketException: null
at (Coroutine boundary.()
•••
Caused by: io.ktor.client.plugins.websocket.WebSocketException: null
at io.ktor.client.engine.java.JavaHttpWebSocket.onError(JavaHttpWebSocket.kt:186)
And the chain of causes stops there. See this build scan for a full example stack trace.
Could you please add the cause here to the WebSocketException
?
Add `append(String, List<String>)` overload to `FormBuilder`
I've stumbled upon improper code generated from swagger spec, roughly equivalent to:
append("key[]", listOf("value0", "value1", "etc"))
Although it is easy to convert to something like:
listOf("value0", "value1", "etc").forEach { append("key[]", it) }
Seems presence of catch-all append<T>
is confusing users:
/**
* Appends a pair [key]:[value] with optional [headers].
*/
@InternalAPI
public fun <T : Any> append(key: String, value: T, headers: Headers = Headers.Empty) {
parts += FormPart(key, value, headers)
}
I propose adding additional append
overload to support List<String>
and optionally List<Number>
parameters:
/**
* Appends a pair [key]:[values] with optional [headers].
*/
public fun append(key: String, values: List<String>, headers: Headers = Headers.Empty) {
require(key.endsWith("[]")) {
"Array parameter must be suffixed with square brackets ie `key[]`"
}
values.forEach { value ->
parts += FormPart(key, value, headers)
}
}
Support for CURLOPT_CAINFO and CURLOPT_CAPATH in ktor-client-curl
We are building a Kotlin Native project for desktop. We decided to use ktor-client-curl, as it supports both Mac and Windows. On the other hand, currently the client fails on Windows with the following exception when using any HTTPS URLs:
io.ktor.client.engine.curl.CurlIllegalStateException: TLS verification failed for request: CurlRequestData(url='https://google.com', method='GET', content: 0 bytes). Reason: SSL peer certificate or SSH remote key was not OK
io.ktor.client.engine.curl.CurlIllegalStateException: TLS verification failed for request: CurlRequestData(url='https://google.com', method='GET', content: 0 bytes). Reason: SSL peer certificate or SSH remote key was not OK
at kotlin.Throwable#<init>(Unknown Source)
at kotlin.Exception#<init>(Unknown Source)
at kotlin.RuntimeException#<init>(Unknown Source)
at kotlin.IllegalStateException#<init>(Unknown Source)
at io.ktor.client.engine.curl.CurlIllegalStateException#<init>(Unknown Source)
at io.ktor.client.engine.curl.internal.CurlMultiApiHandler.collectFailedResponse#internal(Unknown Source)
at io.ktor.client.engine.curl.internal.CurlMultiApiHandler.processCompletedEasyHandle#internal(Unknown Source)
at io.ktor.client.engine.curl.internal.CurlMultiApiHandler.handleCompleted#internal(Unknown Source)
at io.ktor.client.engine.curl.internal.CurlMultiApiHandler#perform(Unknown Source)
at io.ktor.client.engine.curl.CurlProcessor.$runEventLoop$lambda$3COROUTINE$17.invokeSuspend#internal(Unknown Source)
at kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(Unknown Source)
at kotlinx.coroutines.DispatchedTask#run(Unknown Source)
at io.ktor.util.MultiWorkerDispatcher.$workerRunLoop$lambda$4COROUTINE$80.invokeSuspend#internal(Unknown Source)
at kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(Unknown Source)
at kotlinx.coroutines.DispatchedTask#run(Unknown Source)
at kotlinx.coroutines.EventLoopImplBase#processNextEvent(Unknown Source)
at kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal(Unknown Source)
at kotlinx.coroutines#runBlocking(Unknown Source)
It looks like it's not using the certificate store of Windows, and for me it also failed to detect the ca-cert.pem file regardless where I copied it (see https://github.com/ktorio/ktor/issues/671#issuecomment-619832512 for suggestions).
It would be great if the CURLOPT_CAINFO
and CURLOPT_CAPATH
options would be configurable for the Ktor Curl client as this way I could directly provide the path to the ca-cert.pem file.
ByteBufferChannel throws exception when HttpCache, ContentEncoding and onDownload are used
Description
After updating from KTOR 2.1.2 to 2.2.2 and adding HttpCache
I started to receive bug reports from my testers regarding a GZIP magic invalid: 12345
error. After some testing I realized that only release builds but not debug builds were affected and traced the issue to the LogLevel
used, which is LogLevel.INFO
for release and LogLevel.ALL
for debug. Also LogLevel.HEADERS
is affected by this issue.
Updating to KTOR 2.2.3 did not bring any improvements.
Setup
- KTOR 2.2.2 or 2.2.3
- OkHttp Engine on Android is broken
- Darwin Engine on iOS works as expected
The following configuration triggers the issue. It happens with and without setting the public/private storage set to my storage implementation. To ensure my storage doesn't cause the issue, I commented it out with the same result. Setting one or the other or both makes no difference.
To fix the problem, one of three things needs to happen:
- Remove the
install(HttpCache)
, commenting out the storages as shown does not help - Remove the
install(ContentEncoding)
, maybe removing GZIP is sufficient, but the server doesn't provide Deflate so I can't test this - Set
level
toLogLevel.ALL
install(HttpCache) {
// publicStorage(storage)
// privateStorage(storage)
}
install(ContentEncoding) {
deflate(1.0f)
gzip(0.9f)
}
install(Logging) {
level = LogLevel.INFO
logger = object : Logger {
override fun log(message: String) {
println(message)
}
}
}
Stack trace
java.lang.IllegalStateException: GZIP magic invalid: 958
at java.lang.reflect.Constructor.newInstance0(Native Method)
at java.lang.reflect.Constructor.newInstance(Constructor.java:343)
at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$1.invoke(ExceptionUtilsJvm.kt:3)
at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$1.invoke(ExceptionUtilsJvm.kt:1)
at io.ktor.utils.io.ExceptionUtilsJvmKt.tryCopyException(ExceptionUtilsJvm.kt:77)
at io.ktor.utils.io.ByteBufferChannelKt.rethrowClosed(ByteBufferChannel.kt:1)
at io.ktor.utils.io.ByteBufferChannelKt.access$rethrowClosed(ByteBufferChannel.kt:1)
at io.ktor.utils.io.ByteBufferChannel.setupStateForRead(ByteBufferChannel.kt:39)
at io.ktor.utils.io.ByteBufferChannel.readAsMuchAsPossible(ByteBufferChannel.kt:34)
at io.ktor.utils.io.ByteBufferChannel.readAvailable$suspendImpl(ByteBufferChannel.kt:1)
at io.ktor.utils.io.ByteBufferChannel.readAvailable(ByteBufferChannel.kt:3)
at io.ktor.utils.io.jvm.javaio.InputAdapter$loop$1.loop(Blocking.kt:126)
at io.ktor.utils.io.jvm.javaio.InputAdapter$loop$1$loop$1.invokeSuspend(Blocking.kt:12)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:12)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:119)
at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:11)
at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:28)
at io.ktor.utils.io.jvm.javaio.BlockingAdapter.submitAndAwait(Blocking.kt:16)
at io.ktor.utils.io.jvm.javaio.BlockingAdapter.submitAndAwait(Blocking.kt:3)
at io.ktor.utils.io.jvm.javaio.InputAdapter.read(Blocking.kt:6)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:248)
at java.io.BufferedInputStream.read(BufferedInputStream.java:267)
// my app stack
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:12)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:119)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:13)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:3)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:1)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:15)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:29)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:1)
Caused by: java.lang.IllegalStateException: GZIP magic invalid: 958
at io.ktor.util.EncodersJvmKt$inflate$1.invokeSuspend(EncodersJvm.kt:666)
at io.ktor.util.EncodersJvmKt$inflate$1.invoke(EncodersJvm.kt:1)
at io.ktor.util.EncodersJvmKt$inflate$1.invoke(EncodersJvm.kt:2)
at io.ktor.utis.io.CoroutinesKt$launchChannel$job$1.invokeSuspend(Coroutines.kt:68)
... 8 more
While playing around with logger settings I also got this error. It seems like the file I download determines which of the two exceptions I get. Downloading the same file multiple times consistently reproduces the same exception, across app restarts.
java.lang.IllegalArgumentException: max shouldn't be negative: -22207
at java.lang.reflect.Constructor.newInstance0(Native Method)
at java.lang.reflect.Constructor.newInstance(Constructor.java:343)
at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$1.invoke(ExceptionUtilsJvm.kt:103)
at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$1.invoke(ExceptionUtilsJvm.kt:90)
at io.ktor.utils.io.ExceptionUtilsJvmKt.tryCopyException(ExceptionUtilsJvm.kt:66)
at io.ktor.utils.io.ByteBufferChannelKt.rethrowClosed(ByteBufferChannel.kt:2400)
at io.ktor.utils.io.ByteBufferChannelKt.access$rethrowClosed(ByteBufferChannel.kt:1)
at io.ktor.utils.io.ByteBufferChannel.setupStateForRead(ByteBufferChannel.kt:295)
at io.ktor.utils.io.ByteBufferChannel.readAsMuchAsPossible(ByteBufferChannel.kt:2459)
at io.ktor.utils.io.ByteBufferChannel.readAvailable$suspendImpl(ByteBufferChannel.kt:672)
at io.ktor.utils.io.ByteBufferChannel.readAvailable(Unknown Source:0)
at io.ktor.utils.io.jvm.javaio.InputAdapter$loop$1.loop(Blocking.kt:38)
at io.ktor.utils.io.jvm.javaio.InputAdapter$loop$1$loop$1.invokeSuspend(Unknown Source:14)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at io.ktor.utils.io.jvm.javaio.UnsafeBlockingTrampoline.dispatch(Blocking.kt:313)
at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:201)
at io.ktor.utils.io.jvm.javaio.BlockingAdapter.submitAndAwait(Blocking.kt:231)
at io.ktor.utils.io.jvm.javaio.BlockingAdapter.submitAndAwait(Blocking.kt:201)
at io.ktor.utils.io.jvm.javaio.InputAdapter.read(Blocking.kt:65)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:248)
at java.io.BufferedInputStream.read(BufferedInputStream.java:267)
// my app stack
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.lang.IllegalArgumentException: max shouldn't be negative: -22207
at io.ktor.utils.io.ByteBufferChannel.discard$suspendImpl(ByteBufferChannel.kt:1667)
at io.ktor.utils.io.ByteBufferChannel.discard(Unknown Source:0)
at io.ktor.util.EncodersJvmKt$inflate$1.invokeSuspend(EncodersJvm.kt:163)
at io.ktor.util.EncodersJvmKt$inflate$1.invoke(Unknown Source:8)
at io.ktor.util.EncodersJvmKt$inflate$1.invoke(Unknown Source:4)
at io.ktor.utils.io.CoroutinesKt$launchChannel$job$1.invokeSuspend(Coroutines.kt:134)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Response of the server
Following file is downloaded
CE3PRO_pikachu_1gen_flowalistik.gcode
{width=70%}
Logging: Add filter/sanitization of sensitive headers
Extracted this comment to a dedicated issue: https://youtrack.jetbrains.com/issue/KTOR-481/Ktor-client-logging-Add-an-option-to-filter-logged-calls#focus=Comments-27-6332247.0-0
I would like to have the ability to log debug headers like request-id trace headers while not logging sensitive Bearer token headers
CIO: Client fails to parse response without Content-Length, Connection headers and chunked transfer encoding
I'm trying to use Ktor to talk to the REST Api of Shelly Smart Home devices, which use an lwIP webserver.
With a simple client set up like follows:
val httpClient = HttpClient(CIO)
any simple GET request fails with IllegalStateException
:
Failed to parse request body: request body length should be specified,
chunked transfer encoding should be used or
keep-alive should be disabled (connection: close)
because the server neither sends Content-Length
, nor Connection
headers. In fact it doesn't send anything but Content-Type
and Server
:
HTTP/1.0 200 OK
Server: lwIP/2.1.2 (http://savannah.nongnu.org/projects/lwip)
Content-Type: application/json
I understand that this service should probably "behave better", but that's out of my control unfortunately. I debugged into io.ktor.http.cio.HttpBody.parseHttpBody
and it seems to be that the issue would be solvable by defaulting to Connection: close
if no Connection
Header is given; and it seems this is (at least according to wikipedia) in HTTP/1.0 what the default should be; only in HTTP/1.1 connections default to being persistent.
Thanks!
Consider quoting `Boolean` during construction of multipart requests
Currently Boolean
is not supported as a value for form data. However it could be treated the same way as Number
is treated.
Core
Native: Read from a closed socket doesn't throw an exception
Ktor 2.1.3
package transport.ktor
import io.ktor.network.selector.*
import io.ktor.network.sockets.*
import kotlinx.coroutines.*
import kotlin.coroutines.*
import kotlin.random.*
import kotlin.test.*
class KtorReadingFromClosedSocketBugTest {
@Test
fun `Native - Read from a closed socket doesn't throw an exception`() {
val address = InetSocketAddress("localhost", Random.nextInt(2_000..65_000))
val tcp = aSocket(SelectorManager(EmptyCoroutineContext)).tcp()
val serverSocket = tcp.bind(address)
runBlocking {
launch {
serverSocket.accept()
println("client connected")
}
val clientSocket = tcp.connect(address)
launch {
val clientReadChannel = clientSocket.openReadChannel()
println("clientSocket.isClosed: ${clientSocket.isClosed}")
println("clientReadChannel.isClosedForRead: ${clientReadChannel.isClosedForRead}")
delay(2_000)
println("clientSocket.isClosed: ${clientSocket.isClosed}")
println("clientReadChannel.isClosedForRead: ${clientReadChannel.isClosedForRead}")
try {
println("trying to read from clientReadChannel")
clientReadChannel.readByte()
println("reading from clientReadChannel succeeded^")
} catch (e: Exception) {
println("reading from clientReadChannel failed")
}
}
delay(1_000)
clientSocket.close()
println("clientSocket closed")
}
/*
jvm target (expected behavior) native target (erroneous behavior)
------------------------------ ----------------------------------
client connected client connected
clientSocket.isClosed: false clientSocket.isClosed: false
clientReadChannel.isClosedForRead: false clientReadChannel.isClosedForRead: false
clientSocket closed clientSocket closed
clientSocket.isClosed: true clientSocket.isClosed: false
clientReadChannel.isClosedForRead: true clientReadChannel.isClosedForRead: false
trying to read from clientReadChannel trying to read from clientReadChannel
reading from clientReadChannel failed <reading from clientReadChannel never returns>
*/
}
}
Migrate to the new Kotlin JS IR backend
in 1.9.0 K/JS legacy compiler is deprecated, so Ktor is broken in user projects:
e: Old Kotlin/JS compiler is no longer supported. Please migrate to the new JS IR backend
Update Parameters and Headers DSL to be consistent with stdlib
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1091
Ktor Version
ktor-http
1.1.5 and below
Feedback
Add a parameters { ... }
builder DSL to replace the less discoverable Parameters.build { ... }
companion object function in favor of being more consistent with the stdlib.
sequenceOf(1, 2, 3)
/listOf(1, 2, 3)
-> parametersOf("first", "a")
sequence { ... }
-> parameters { ... }
Docs
Document how to run a Native server app
Generator
New project with JWT plugin fails to start
/home/leonid/.jdks/corretto-11.0.16/bin/java -javaagent:/home/leonid/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/222.3345.118/lib/idea_rt.jar=44027:/home/leonid/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/222.3345.118/bin -Dfile.encoding=UTF-8 -classpath /home/leonid/work/js/frameworks-demo/ktor-sample/build/classes/kotlin/main:/home/leonid/work/js/frameworks-demo/ktor-sample/build/resources/main:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-content-negotiation-jvm/2.1.0/8146ef5e0b878e53003ec6c7dd8993c593e426e6/ktor-server-content-negotiation-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-auth-jwt-jvm/2.1.0/aa5373da09ad259b9483c6d727e4e3e0ffb948fd/ktor-server-auth-jwt-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-auth-jvm/2.1.0/3e54cc1f56cc79f4a52070fed610d057b7446362/ktor-server-auth-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-netty-jvm/2.1.0/dfa95e58d34d851bb80df97eef7edebcf58fa07c/ktor-server-netty-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-core-jvm/2.1.0/48db9240b2424eaadebedf06cb4b8a23badd4870/ktor-server-core-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-serialization-kotlinx-json-jvm/2.1.0/f443b9bc5f2c966026f71ab58c4afad86bf74ee5/ktor-serialization-kotlinx-json-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk8/1.7.10/d70d7d2c56371f7aa18f32e984e3e2e998fe9081/kotlin-stdlib-jdk8-1.7.10.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/ch.qos.logback/logback-classic/1.2.3/7c4f3c474fb2c041d8028740440937705ebb473a/logback-classic-1.2.3.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk7/1.7.10/1ef73fee66f45d52c67e2aca12fd945dbe0659bf/kotlin-stdlib-jdk7-1.7.10.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-jdk8/1.6.4/5bc4b0bf6fd90fc190fd2f17e919c74c6274cb71/kotlinx-coroutines-jdk8-1.6.4.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.36/6c62681a2f655b49963a5983b8b0950a6120ae14/slf4j-api-1.7.36.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-common/1.7.10/bac80c520d0a9e3f3673bc2658c6ed02ef45a76a/kotlin-stdlib-common-1.7.10.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.auth0/java-jwt/3.19.2/497c48847dcc1074d1060f9be08166aaf0853764/java-jwt-3.19.2.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.auth0/jwks-rsa/0.17.0/7c0fcd847940b6776163ef3c677d92c8e4b36798/jwks-rsa-0.17.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.googlecode.json-simple/json-simple/1.1.1/c9ad4a0850ab676c5c64461a05ca524cdfff59f1/json-simple-1.1.1.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http2/4.1.78.Final/9a4e44bebe5979b59c9bf4cd879e65c4c52bf869/netty-codec-http2-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.eclipse.jetty.alpn/alpn-api/1.1.3.v20160715/a1bf3a937f91b4c953acd13e8c9552347adc2198/alpn-api-1.1.3.v20160715.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-kqueue/4.1.78.Final/7a48e5f1cce660dd4fb5a32d9cf68d731f969555/netty-transport-native-kqueue-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-epoll/4.1.78.Final/dcb28ed02abf7bf02aa8dc9bf05f58bedcb12e9a/netty-transport-native-epoll-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-reflect/1.7.0/e660b0079fdaf744dc9cc7f6f3aa3c761ec839b0/kotlin-reflect-1.7.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.typesafe/config/1.4.2/4c40a633e7994cfb0354244efb6d03fcb11c3ecf/config-1.4.2.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.7.10/d2abf9e77736acc4450dc4a3f707fa2c10f5099d/kotlin-stdlib-1.7.10.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/ch.qos.logback/logback-core/1.2.3/864344400c3d4d92dfeb0a305dc87d953677c03c/logback-core-1.2.3.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-core-jvm/1.6.4/2c997cd1c0ef33f3e751d3831929aeff1390cb30/kotlinx-coroutines-core-jvm-1.6.4.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-sessions-jvm/2.1.0/aea94045c1ebe7a7c0218bbcb694d9ff9a9ae12c/ktor-server-sessions-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-client-core-jvm/2.1.0/9f817ae55e463f27d885ef051ff88e4c2acea7da/ktor-client-core-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-server-host-common-jvm/2.1.0/4f8a7ed4d46d72f78d7eb806d31863e7e6a99626/ktor-server-host-common-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http/4.1.78.Final/b636591e973418479f2d08e881b12be61845aa6f/netty-codec-http-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler/4.1.78.Final/7e902b6018378bb700ec9364b1d0fba6eefd99fd/netty-handler-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.78.Final/e8486c8923fc0914df4562a8e0923462e885f88a/netty-codec-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.78.Final/b1639d431e43622d6cbfdd45c30d3fb810fa9101/netty-transport-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.78.Final/6c0c2d805a2c8ea30223677d8219235f9ec14c38/netty-buffer-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.78.Final/d6f560098050f88ba11750aa856edd955e4a7707/netty-common-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-classes-kqueue/4.1.78.Final/5536de8030ef84507504b352e54c298b9db361c9/netty-transport-classes-kqueue-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-unix-common/4.1.78.Final/62c64e182a11e31ad80e68a94f0b02b45344df23/netty-transport-native-unix-common-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-classes-epoll/4.1.78.Final/2151893a521c6362858071053fddbd270c1f20bd/netty-transport-classes-epoll-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-serialization-jvm/2.1.0/e1d36c91035d2eddd1f202411248a4b6944ea387/ktor-serialization-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-events-jvm/2.1.0/ef58c3c3d012ab115011af1e2f2afc357433c220/ktor-events-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-http-jvm/2.1.0/eff494019c2c1a0e9db6e13533d241ecd6bbc822/ktor-http-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-utils-jvm/2.1.0/1b3ca53f817d054ba00c8a3e92a50ba655428c71/ktor-utils-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-serialization-kotlinx-jvm/2.1.0/c852a1ac0196c9b3e9085b7d19e515f3774025f6/ktor-serialization-kotlinx-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-serialization-json-jvm/1.3.3/6298c404c1159685fb657f15fa5b78630da58c96/kotlinx-serialization-json-jvm-1.3.3.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains/annotations/13.0/919f0dfe192fb4e063e7dacadee7f8bb9a2672a9/annotations-13.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.78.Final/691170d70a979757d50f60e16121ced5f411a9b8/netty-resolver-4.1.78.Final.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-websocket-serialization-jvm/2.1.0/d202979642d5091a18587b3f81269c0a9a8acc78/ktor-websocket-serialization-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-websockets-jvm/2.1.0/17015d23235fd5e784230875e630f6bd1418fdf8/ktor-websockets-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-http-cio-jvm/2.1.0/81fc128a33b4cdf20ec48462d165d071f366397e/ktor-http-cio-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-io-jvm/2.1.0/ccd958db554677538c21515a0acd62862bb6b2a0/ktor-io-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-serialization-core-jvm/1.3.3/3c22103b5d8e3c262db19e85d90405ca78b49efd/kotlinx-serialization-core-jvm-1.3.3.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/io.ktor/ktor-network-jvm/2.1.0/e1a24aa8ef91988a7d33ba95b51f21b3cfc334a4/ktor-network-jvm-2.1.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.fusesource.jansi/jansi/2.4.0/321c614f85f1dea6bb08c1817c60d53b7f3552fd/jansi-2.4.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.13.2.2/ffeb635597d093509f33e1e94274d14be610f933/jackson-databind-2.13.2.2.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.15/49d94806b6e3dc933dacbd8acb0fdbab8ebd1e5d/commons-codec-1.15.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/30.0-jre/8ddbc8769f73309fe09b54c5951163f10b0d89fa/guava-30.0-jre.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.13.2/ec18851f1976d5b810ae1a5fcc32520d2d38f77a/jackson-annotations-2.13.2.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.13.2/a6a0e0620d51833feffc67bccb51937b2345763/jackson-core-2.13.2.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.google.guava/failureaccess/1.0.1/1dcf1de382a0bf95a3d8b0849546c88bac1292c9/failureaccess-1.0.1.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.google.guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/b421526c5f297295adef1c886e5246c39d4ac629/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.checkerframework/checker-qual/3.5.0/2f50520c8abea66fbd8d26e481d3aef5c673b510/checker-qual-3.5.0.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.google.errorprone/error_prone_annotations/2.3.4/dac170e4594de319655ffb62f41cbd6dbb5e601e/error_prone_annotations-2.3.4.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/1.3/ba035118bc8bac37d7eff77700720999acd9986d/j2objc-annotations-1.3.jar:/home/leonid/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-slf4j/1.6.4/9369fcabacb66a55ca3b7939b72bb0b62af112ce/kotlinx-coroutines-slf4j-1.6.4.jar com.example.ApplicationKt
2022-08-12 10:28:22.860 [main] INFO ktor.application - Autoreload is disabled because the development mode is off.
Exception in thread "main" io.ktor.server.config.ApplicationConfigurationException: Property jwt.audience not found.
at io.ktor.server.config.MapApplicationConfig.property(MapApplicationConfig.kt:50)
at com.example.plugins.SecurityKt$configureSecurity$1$1.invoke(Security.kt:17)
at com.example.plugins.SecurityKt$configureSecurity$1$1.invoke(Security.kt:16)
at io.ktor.server.auth.jwt.JWTAuthKt.jwt(JWTAuth.kt:326)
at io.ktor.server.auth.jwt.JWTAuthKt.jwt$default(JWTAuth.kt:322)
at com.example.plugins.SecurityKt$configureSecurity$1.invoke(Security.kt:16)
at com.example.plugins.SecurityKt$configureSecurity$1.invoke(Security.kt:15)
at io.ktor.server.auth.Authentication$Companion.install(Authentication.kt:98)
at io.ktor.server.auth.Authentication$Companion.install(Authentication.kt:94)
at io.ktor.server.application.ApplicationPluginKt.install(ApplicationPlugin.kt:98)
at io.ktor.server.auth.AuthenticationKt.authentication(Authentication.kt:122)
at com.example.plugins.SecurityKt.configureSecurity(Security.kt:15)
at com.example.ApplicationKt$main$1.invoke(Application.kt:10)
at com.example.ApplicationKt$main$1.invoke(Application.kt:8)
at io.ktor.server.engine.internal.CallableUtilsKt.executeModuleFunction(CallableUtils.kt:51)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$launchModuleByName$1.invoke(ApplicationEngineEnvironmentReloading.kt:334)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$launchModuleByName$1.invoke(ApplicationEngineEnvironmentReloading.kt:333)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartupFor(ApplicationEngineEnvironmentReloading.kt:358)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.launchModuleByName(ApplicationEngineEnvironmentReloading.kt:333)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.access$launchModuleByName(ApplicationEngineEnvironmentReloading.kt:32)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:321)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:312)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartup(ApplicationEngineEnvironmentReloading.kt:340)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.instantiateAndConfigureApplication(ApplicationEngineEnvironmentReloading.kt:312)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createApplication(ApplicationEngineEnvironmentReloading.kt:149)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.start(ApplicationEngineEnvironmentReloading.kt:279)
at io.ktor.server.netty.NettyApplicationEngine.start(NettyApplicationEngine.kt:210)
at com.example.ApplicationKt.main(Application.kt:12)
at com.example.ApplicationKt.main(Application.kt)
Process finished with exit code 1
IntelliJ IDEA Plugin
Extracting to application module quickfix produces non-compilable code when CLI args are captured
To reproduce, write the following code:
import io.ktor.server.application.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import io.ktor.server.response.*
import io.ktor.server.routing.*
fun main(args: Array<String>) {
val x = args.firstOrNull()
embeddedServer(Netty, port = 8888) {
routing {
get("/") {
call.respondText { x.toString() }
}
}
}.start(wait = true)
}
And then apply the Extract Application Module from the current emdeddedServer block
quick fix.
As a result, the generated code isn't compilable:
fun main(args: Array<String>) {
val x = args.firstOrNull()
embeddedServer(Netty, port = 8888, module = Application::myApplicationModule).start(wait = true)
}
fun Application.myApplicationModule() {
routing {
get("/") {
call.respondText { x.toString() } // Unresolved reference: x
}
}
}
OpenAPI: `respondRedirect` (permanent and non-permanent) result in unresolved status code
call.respondRedirect("https://www.google.com")
Note: in Ktor, this function is implemented as following:
public suspend fun ApplicationCall.respondRedirect(url: String, permanent: Boolean = false) {
response.headers.append(HttpHeaders.Location, url)
respond(if (permanent) HttpStatusCode.MovedPermanently else HttpStatusCode.Found)
}
Expected: if permanent = false, HttpStatusCode.MovedPermanently
should be the status code, and HttpStatusCode.Found
otherwise.
Actual: no information in OpenAPI file
Ktor modules are not resolved from tests module's application.yaml
While it works fine in main module, if you add a separate YAML config for tests, that's what you'll see:
OpenAPI: Support extensions for unhandled exceptions inside StatusPages
intall(StatusPages) {
exception<E> { ... }
unhandledException()
}
fun StatusPagesConfig.unhandledException() {
call.respond(...)
}
Here, call.respond
from unhandledException
must be present in OpenAPI as well.
OpenAPI: schema type does not always contain reference type if more than one `call.respond` calls is used
Example:
get("/a/b/c") {
call.respond(randomCoordinate())
exampleRespond(Coordinate(10, 20))
}
class Coordinate(val x: Int, val y: Int)
fun randomCoordinate(): Coordinate = Coordinate(Random.nextInt(), Random.nextInt())
fun Route.exampleRespond(value: Any?) {
// A hack to add OpenAPI examples without affection actuall HTTP behaviour:
if (false) {
call.respond(value)
}
}
Expected: OpenAPI shows that /a/b/c
route has #ref/Coordinate
response type reference.
(Ideally, it also has to show value: x: 10 y: 20
as an example value for that response type, but it is not supported yet (KTOR-5683))
Actual: OpenAPI shows that /a/b/c
route has an object
response type.
OpenAPI generation can not be canceled (in case of larger projects it takes some time and cancelation must be possible)
Example: Open Qodana Cloud project and generate OpenAPI for core-api
module.
Computation will run for around 1 minute without an option to cancel.
OpenAPI: support examples for object types
Example:
get("/a/b/c") {
call.respond(Coordinate(10, 20))
}
class Coordinate(val x: Int, val y: Int)
Expected: OpenAPI has to show value: x: 10 y: 20
as an example value for that response type of /a/b/c
request.
Actual: example it is not reflected in OpenAPI.
OpenAPI: Support arrays in example values
Example:
call.respond(MyData(x = 1, y = 2, items = listOf("a", "b", "c")))
Or even:
call.respond(MyData(x = 1, y = 2))
...
class MyData(val x: Int, val y: Int, val items: List<String> = listOf("a", "b", "c"))
OpenAPI should have an example entrance:
examples:
Example#1:
description: ""
value:
x: 1
y: 1
items:
- "a"
- "b"
- "c"
OpenAPI: Analyze examples passed via function calls (support data flow analysis)
Example:
routing {
get("/my/route") {
sendValue(MyObject(1, 2))
}
}
...
fun <T> PipelineContext<Unit, ApplicationCall>.sendValue(value: T) {
call.respond(HttpStatusCode.Accepted, value)
}
Expected: After generating OpenAPI documentation, it should tell about MyObject(1, 2) response with example value based on what was passed to sendValue
function.
OpenAPI: deeply analyze generic types of responses
Example:
data class PaginatedList<T>(
@Expose val count: Long,
@Expose val next: Long?,
@Expose val prev: Long?,
@Expose val items: List<T>,
)
data class ProjectListResponseItem(
@Expose val id: String,
@Expose val name: String,
@Expose val slug: String = name.asSlug,
@Expose val branch: String? = null,
@Expose val commit: String? = null,
@Expose val lastChecked: Instant? = null,
@Expose @NoSlug val lastReportId: String? = null,
@Expose val lastState: ReportState?,
@Expose val url: String? = null,
@Expose val problems: Problems? = null,
@Expose val baselineCount: Int? = null,
)
...
call.respond(results())
...
fun results(): PaginatedList<ProjectListResponseItem>
Expected:
responses:
"200":
description: ""
content:
'*/*':
schema:
$ref: "#/components/schemas/PaginatedList_ProjectListResponseItem"
...
schemas:
PaginatedList_ProjectListResponseItem:
type: "object"
properties:
count:
type: "integer"
format: "int64"
next:
type: "integer"
format: "int64"
prev:
type: "integer"
format: "int64"
items:
type: "array"
items:
$ref: "#/components/schemas/ProjectListResponseItem"
ProjectListResponseItem:
type: "object"
properties:
id:
type: "string"
name:
type: "string"
slug:
type: "string"
branch:
type: "string"
commit:
type: "string"
lastChecked:
$ref: "#/components/schemas/Instant"
lastReportId:
type: "string"
lastState:
type: "string"
enum:
- "UPLOADING"
- "UPLOADED"
- "PROCESSED"
- "FAILED_TO_PROCESS"
- "PINNED"
- "MARKED_FOR_DELETION"
- "OVERWRITTEN"
url:
type: "string"
problems:
$ref: "#/components/schemas/Problems"
baselineCount:
type: "integer"
format: "int32"
OpenAPI: statusCode and other fields from exceptions are not represented in OpenAPI
Example:
There is a StatusPages plugin that handles exceptions:
exception<MyExceptionType> { call, cause ->
val descriptor = ApiDescr(cause.name, cause.message)
call.respond(cause.statusCode, descriptor) // <--- HERE
}
And routes:
get("/first/route") {
throw MyExceptionType1("first name", "Cool message!")
}
get("/second/route") {
throw MyExceptionType2("2nd name", "Bad message :(")
}
And exceptions:
class MyExceptionType(val statusCode: Int, val type: String, val details: String)
class MyExceptionType1:(type: String, details: String): MyExceptionType(HttpStatusCode.BAD_REQUEST, type, details)
class MyExceptionType2:(type: String, details: String): MyExceptionType(HttpStatusCode.UNAUTHORIZED, type, details)
Expected OpenAPI:
/first/route:
get:
description: ""
responses:
"400":
description: "Bad Request"
content:
'*/*':
schema:
$ref: "#/components/schemas/ApiDescr"
examples:
Example#1:
value:
type: "first name"
details: "Cool message!"
/second/route:
get:
description: ""
responses:
"401":
description: "Unauthorized"
content:
'*/*':
schema:
$ref: "#/components/schemas/ApiDescr"
examples:
Example#1:
value:
type: "2nd name"
details: "Bad message :("
components:
schemas:
APIError:
type: "object"
properties:
type:
name: "string"
details:
message: "string"
Actual OpenAPI: nothing is known about response types at all.
Note: even if status codes are known statically (written in exception<...>
handlers), the APIError messages can not be deduced from exceptions.
Modules included from application.conf / application.yaml must be present in OpenAPI
Example:
ktor:
application:
modules:
- a.b.c.myModule1
- a.b.c.myModule2
fun Application.myModule1() {
// routes
}
fun Application.myModule2() {
// status pages
}
Expected: In this case, all exceptions thrown from myModule1
must be handled by StatusPages defined in myModule2
. And this must be represented in OpenAPI.
Actual: OpenAPI generator does not know that myModule2
is also included to the application configuration and hence it does not analyze StatusPages from there because it is not directly referenced.
OpenAPI generator has to check application configuration files to know which other modules it has to analyze.
Endpoints view throws SOE when there are recursive Routing calls
Example:
fun Application.main() {
routing {
f()
}
}
fun Route.f() {
get("/f") {...}
g()
}
fun Route.g() {
route("/g") {
f()
}
}
Expected: at least routes /f
and g/f
are shown.
Actual: no routes are shown, StackOverflow is thrown.
Application method references in YAML are not resolvable if `Application` is imported without wildcard in .kt file
Example:
package a.b.c
import io.ktor.server.Application
fun Application.myModule() { ... }
yaml:
ktor:
application:
modules:
- a.b.c.myModule
Expected: a.b.c.myModule
is a reference in yaml leading to Application.myModule()
.
Actual: it does not resolve.
YAML: Support enviroment variables in YAML config
Example:
- Without default values:
ktor:
deployment:
port: $PORT
- And with default values:
ktor:
deployment:
port: "$PORT:8080"
Expected highlighting:
{width=70%}
Actual: currently, the whole "$PORT:8080" is seen as one single string that is a candidate to be converted to Integer, because variables are not supported.
Search results in wizards doesn't work with spaces / names for plugins should use spaces
Searching for "Content Negotiation" both in the web and IntelliJ wizards gives the following results:
I suppose this is because ContentNegotiation
doesn't contain a space (even though other features do).
I suggest:
- Renaming the feature to
Content Negotiation
- Ignoring whitespace in the fuzzy search.
Support WebSockets endpoints in the Endpoints tool window
Documentation for the Endpoints tool window says that WebSocket endpoints are supported:
The Endpoints tool window provides an aggregated view of both client and server APIs used in your project for HTTP and WebSocket protocols.
It would be useful to have this capability for Ktor projects. Currently, only HTTP routes are displayed.
Server
DoubleRecieve throws exception with CallLogging
io.ktor.request.RequestAlreadyConsumedException: Request body has been already consumed (received).
at io.ktor.features.DoubleReceive$Feature$install$1.invokeSuspend(DoubleReceive.kt:94)
embeddedServer(Jetty, 8080) {
install(ContentNegotiation) {
json(json = json)
}
install(DoubleReceive) {
this.receiveEntireContent = true
}
install(CallLogging) {
format {
runBlocking {
it.receiveText()
}
}
}
}
Support Flow in ktor-serialization
I was surprised that kotlinx.coroutines Flow<T>
is currently not supported by ktor-serialization, Flow is not collected so nothing happens.
What steps will reproduce the issue?
This is a minimal reproducer
fun main(args: Array<String>) {
embeddedServer(Netty, 8080) {
install(ContentNegotiation) {
json()
}
routing {
get("/") {
call.respond(
flow {
emit(1)
delay(1000)
emit(2)
}
)
}
}
}.start(wait = true)
}
What is expected
As coroutines are heavilly used and supported in Ktor, I think ktor-serialization should support kotlinx.coroutines Flow<T>
natively, as it is the official asynchronous Stream on Kotlin coroutines world.
I don't think that the Flow<T>
support should require an additional Plugin, but you can of course disagre with my opinion ;)
The minimal support (on multiplatform) can be to just collect the Flow<T>
as a List<T>
, and then serialize it.
Sequence<T>
should be available for deserialization too.
Bonus 1:
On JVM (with kotlinx.serialization, Jackson and Gson) normal and Flow<T>
serialization can be optimized, and asynchronously write in the OutpuStream (server mode) and read from InpuStream (client mode), or using java.io.Writer
/java.io.Reader
. Examples are available here in a kotlinx.serialization JsonLazySequenceTest.kt
I made a custom implementation to support Flow in Ktor server mode in this class on my kotysa-ktor-r2dbc-coroutines sample project -> by the way if you have a few minutes to read this sample's very short code, and tell me if I made a mistake thanks a lot ^^
Bonus 2:
Flow<T>
could also be used to asynchronously serialize data as Server Sent Events and in Websocket in ktor-websocket-serialization
.
The unofficial application/x-json-stream
(MediaType.APPLICATION_JSON_STREAM) could be supported aswell, even if not supported by Browsers and non standard yet, this format can be useful for server-to-server JSON communication. See this micronaut topic
Can't visit files of staticRootFolder
Support preCompressed with resources
/**
* Support pre-compressed files in the file system only (not just any classpath resource)
* ...
public fun Route.preCompressed(
It looks like preCompressed only works with files() at the moment, why ?
Support `100 Continue`
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/119
We have continue support in netty however it shouldn't be done like that: we have to respond continue (to allow client sending request body) only when handler tries to access request body (call.request.receive()
) or at least once request handler has started (that is not exactly correct as user's code could respond 404)
Websockets: Connection should fail immediately (1002/Protocol Error) when control frame has a payload with more than 125 octets
To reproduce run the following test:
class WebsocketsTest {
private val server = embeddedServer(CIO, host = "127.0.0.1", port = 0) {
install(WebSockets)
routing {
webSocket("/") {
for (frame in incoming) {
outgoing.send(frame)
}
}
}
}
private val client = HttpClient(io.ktor.client.engine.cio.CIO) {
install(io.ktor.client.plugins.websocket.WebSockets)
}
@Before
fun setup() {
server.start(wait = false)
}
@After
fun dispose() {
server.stop()
}
@Test
fun test() = testSuspend {
val session = createSession()
session.outgoing.send(Frame.Ping(ByteArray(126)))
val frame = session.incoming.receive()
assertIs<Frame.Close>(frame)
assertEquals(1002, frame.readReason()?.code)
}
private suspend fun createSession(): ClientWebSocketSession {
val url = "ws://127.0.0.1:${server.resolvedConnectors().first().port}/"
return client.webSocketRawSession { url(url) }
}
}
Websockets: Connection should be failed immediately, since all data frames after the initial data frame must have opcode 0
To reproduce run the following test:
class WebsocketsTest {
private val server = embeddedServer(CIO, host = "127.0.0.1", port = 0) {
install(WebSockets)
routing {
webSocket("/") {
for (frame in incoming) {
outgoing.send(frame)
}
}
}
}
private val client = HttpClient(io.ktor.client.engine.cio.CIO) {
install(io.ktor.client.plugins.websocket.WebSockets)
}
@Before
fun setup() {
server.start(wait = false)
}
@After
fun dispose() {
server.stop()
}
@Test
fun test() = testSuspend {
val session = createSession()
session.outgoing.send(Frame.Text(false, "hello ".toByteArray()))
session.outgoing.send(Frame.Text(true, "world".toByteArray()))
val frame = session.incoming.receive()
assertIs<Frame.Close>(frame)
}
private suspend fun createSession(): ClientWebSocketSession {
val url = "ws://127.0.0.1:${server.resolvedConnectors().first().port}/"
return client.webSocketRawSession { url(url) }
}
}
Non-standard `Content-Type` headers for static files
When providing files via a static
block, a non-standard content-type is used.
Code:
fun Application.main() {
install(Routing)
static("assets") {
files("public")
}
}
File structure:
\- public
|- some-html-file.html
|- some-image.png
\- some-ttf-font.ttf
Making requests to the following urls returns a response with the expected Content-Type
header:
/assets/some-html-file.html
returnsContent-Type: text/hml
/assets/some-image.png
returnsContent-Type: image/png
However, making a request to /assets/some-ttf-font.ttf
returns an unexpected Content-Type
.
Example response:
HTTP/1.1 200 OK
Date: Thu, 08 Dec 2022 21:23:03 GMT
Server: Ktor/2.0.3
Last-Modified: Mon, 15 Aug 2022 22:09:01 GMT
Accept-Ranges: bytes
Content-Length: 335196
Content-Type: application/x-font-ttf
According to the IANA, the correct MIME type for a .ttf
file is font/ttf
.
Here is a list of file types which return an incorrect Content-Type
:
ttf
- Expected:
font/ttf
- Actual:
application/x-font-ttf
- Expected:
woff2
- Expected:
font/woff2
- Actual:
application/octet-stream
- Expected:
woff
- Expected:
font/woff
- Actual:
application/x-font-woff
- Expected:
otf
- Expected:
font/otf
- Actual:
application/x-font-otf
- Expected:
gz
- Expected:
application/gzip
- Actual:
application/x-compressed
- Expected:
The source of the issues is the Mimes.kt
file in ktor-http
module.
CallLogging: add config to avoid logging static file request
I want to avoid logging static file requests,currently I avoid logging static file requests like this:
install(CallLogging) {
level = Level.INFO
filter { !it.parameters.contains("static-content-path-parameter") }
}
Since the pathParameterName
property of StaticContent.kt
is private, I have to manually copy the String. Maybe we can make the pathParameterName
String property of StaticContent.kt
public ,or add configuration items in the CallLogging plugin.
The '425 Too Early' status code is missing in the HttpStatusCode enum
Feature request: SO_REUSEADDR option for embedded server
My Environment
iosX64 Native - iOS 16.0 Simulator
Kotlin: 1.8.0
XCode: 14.0.1
Ktor Version: 2.2.3
Background
I'm running a Ktor server for mocking purposes on an iOS simulator. The problem is that when killing the application through XCode, I get no callback to stop the server gracefully (i.e. it's a SIGKILL).
If a request was in progress when the app is killed, it leads to the following exception upon restarting the app (unless I wait about 30 seconds).
io.ktor.utils.io.errors.PosixException.AddressAlreadyInUseException: EADDRINUSE
Feature Request
(I'm a mobile engineer so apologies if what I'm asking is incorrect). Can there be a flag added to the embeddedServer
to allow port reuse, so that killing and restarting the app with the same server port does not lead to a crash?
I.e exposing the API used here to the embeddedServer
configuration: https://youtrack.jetbrains.com/issue/KTOR-4442/EADDRINUSE-on-socket-bind-on-Linux.
Minimal Reproducible Example
https://github.com/SamCosta1/ktor-connection-bug-example/tree/ktor-killing-app-example
Notes
I've not experienced this issue running on Android, this seems to be Native only.
Support loading multiple configuration files
We would like to provide a solution for loading multiple configuration files, i.e. java -jar sample-app.jar -config=application.conf -config=applicationFoo.conf -config=applicationBar.conf.
The mechanism should work from left to right - it means if application.conf and applicationFoo.conf have common properties then values from application.conf should be overwritten by values from applicationFoo.conf, like shown below:
application.conf
ktor {
deployment {
port = 8080
}
application {
class = <org.company.ApplicationClass>
}
}
applicationFoo.conf
ktor {
deployment {
port = 8085
testProperty = "TestValue"
}
}
config {
configurationProperty = "Config value"
}
result configuration:
ktor {
deployment {
port = 8085
testProperty = "TestValue"
}
application {
class = <org.company.ApplicationClass>
}
}
config {
configurationProperty = "Config value"
}
"Serializer for class 'Any' is not found" error when responding with Any type since Ktor 2.2.4
This used to work in 2.2.3:
fun main() {
embeddedServer(Netty, port = 8082, host = "0.0.0.0", module = Application::module)
.start(wait = true)
}
fun Application.module() {
install(ContentNegotiation) {
json()
}
routing {
get {
call.respond(value())
}
}
}
// fun value(): Data = Data("bar") works
fun value(): Any = Data("bar")
@Serializable
data class Data(val foo: String)
on 2.2.4 ktor throws
2023-03-21 01:45:13.069 [eventLoopGroupProxy-4-1] ERROR ktor.application - Unhandled: GET - /
kotlinx.serialization.SerializationException: Serializer for class 'Any' is not found.
Mark the class as @Serializable or provide the serializer explicitly.
CIO: nmap crashes server with "SocketException: Invalid argument" error
so I have a simplest ktor app
fun main() {
val env = applicationEngineEnvironment {
module {
routing {
get("/") {
call.respondText("Hello, world!")
}
}
}
connector {
this.port = 8080
}
}
embeddedServer(CIO, env).start(wait = true)
}
implementation("io.ktor:ktor-server-core:2.2.3")
implementation("io.ktor:ktor-server-cio:2.2.3")
if you perform a scan of open tcp ports like this nmap localhost -Pn -p8080
, the app crashes with
Exception in thread "DefaultDispatcher-worker-2" java.net.SocketException: Invalid argument
at java.base/sun.nio.ch.Net.setIntOption0(Native Method)
at java.base/sun.nio.ch.Net.setSocketOption(Net.java:455)
at java.base/sun.nio.ch.Net.setSocketOption(Net.java:393)
at java.base/sun.nio.ch.SocketChannelImpl.setOption(SocketChannelImpl.java:280)
at io.ktor.network.sockets.ServerSocketImpl.accepted(ServerSocketImpl.kt:51)
at io.ktor.network.sockets.ServerSocketImpl.acceptSuspend(ServerSocketImpl.kt:42)
at io.ktor.network.sockets.ServerSocketImpl.access$acceptSuspend(ServerSocketImpl.kt:12)
at io.ktor.network.sockets.ServerSocketImpl$acceptSuspend$1.invokeSuspend(ServerSocketImpl.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Suppressed: kotlinx.coroutines.DiagnosticCoroutineContextException: [LazyStandaloneCoroutine{Cancelling}@23a3bd4f, Dispatchers.IO]
OAuth2: "JsonObject is not a JsonPrimitive" error when server replies with nested JSON object on a token request
Hi,
I am trying to integrate with a service using OAuth2, however the response that the service gives does not get handled correctly by ktor. From the Asana website, this is a typical response given (https://developers.asana.com/docs/oauth)
{
"access_token": "f6ds7fdsa69ags7ag9sd5a",
"expires_in": 3600,
"token_type": "bearer",
"data": {
"id": 4673218951,
"gid": "4673218951",
"name": "Greg Sanchez",
"email": "gsanchez@example.com"
},
"refresh_token": "hjkl325hjkl4325hj4kl32fjds"
}
Currently ktor expects the response to only contain primitives. When trying to get the response after exchanging the code for the token, I get this error message from ktor.
Element class kotlinx.serialization.json.JsonObject is not a JsonPrimitive
I was wondering if this could be supported, possibly by adding these as extra parameters
Thanks!
CallLogging: logs caused by an exception are suppressed when mdc provider is configured
Ktor version: 2.2.4
Kotlin version: 1.8.10
When running ktor server like:
val server = embeddedServer(Netty, port = 28080) {
install(CallLogging) {
callIdMdc()
}
routing {
get("/test") {
this.call.respondText("Hello, world!")
}
get("/err") {
throw Exception("Unexpected")
}
}
}
It will not log calls to /err route:
@Test
fun testRoot() = runTest {
server.start(false)
val client = HttpClient(CIO)
assertEquals(200, client.get("http://localhost:28080/test").status.value)
assertEquals(500, client.get("http://localhost:28080/err").status.value)
client.close()
server.stop()
}
Produced logs:
2023-03-16 10:14:44,668 INFO [] Application started in 0.284 seconds. [ktor.application] [oroutine#2]
2023-03-16 10:14:44,669 INFO [] Application started: io.ktor.server.application.Application@4f3356c0 [ktor.application] [oroutine#2]
2023-03-16 10:14:45,043 INFO [] Responding at http://0.0.0.0:28080 [ktor.application] [oroutine#1]
2023-03-16 10:14:45,641 INFO [] 200 OK: GET - /test in 163ms [ktor.application] [handler#16]
2023-03-16 10:14:45,669 ERROR [] Unhandled: GET - /err [ktor.application] [handler#29]
java.lang.Exception: Unexpected
(Coroutine boundary)
at io.ktor.util.pipeline.PipelineKt$execute$2.invokeSuspend(Pipeline.kt:478)
at io.ktor.server.routing.Routing.executeResult(Routing.kt:188)
2023-03-16 10:14:47,492 INFO [] Application stopping: io.ktor.server.application.Application@4f3356c0 [ktor.application] [oroutine#2]
2023-03-16 10:14:47,493 INFO [] Application stopped: io.ktor.server.application.Application@4f3356c0 [ktor.application] [oroutine#2]
When CallLogin plug is not configured with mds, it will produces logs:
2023-03-16 10:13:47,070 INFO [] Application started in 0.269 seconds. [ktor.application] [oroutine#2]
2023-03-16 10:13:47,071 INFO [] Application started: io.ktor.server.application.Application@6ac45c0c [ktor.application] [oroutine#2]
2023-03-16 10:13:47,432 INFO [] Responding at http://0.0.0.0:28080 [ktor.application] [oroutine#1]
2023-03-16 10:13:48,031 ERROR [] Unhandled: GET - /err [ktor.application] [handler#29]
java.lang.Exception: Unexpected
(Coroutine boundary)
at io.ktor.util.pipeline.PipelineKt$execute$2.invokeSuspend(Pipeline.kt:478)
at io.ktor.server.routing.Routing.executeResult(Routing.kt:188)
2023-03-16 10:13:48,170 INFO [] 200 OK: GET - /test in 312ms [ktor.application] [handler#16]
2023-03-16 10:13:48,717 INFO [] 500 Internal Server Error: GET - /err in 730ms [ktor.application] [handler#29]
2023-03-16 10:13:49,761 INFO [] Application stopping: io.ktor.server.application.Application@6ac45c0c [ktor.application] [oroutine#2]
2023-03-16 10:13:49,762 INFO [] Application stopped: io.ktor.server.application.Application@6ac45c0c [ktor.application] [oroutine#2]
Static files filters or something similar to mod_rewrite
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/233
I want Ktor to drive my website. I have a bunch of HTML files but I want to strip the HTML suffix from those files when they are being served as the HTML suffix is just a technical detail that I want to strip.
Built-in support for HEAD requests for static files
When serving static files in static
routes using the files(dir)
DSL, it would be nice if a HEAD
mapping was setup in addition to the GET
one, in order to respect HTTP semantics.
The HEAD
mapping should return (among others) the correct status code (e.g. 404 if the file doesn't exist) and also the file size as Content-Length
for instance.
I stumbled upon this issue when trying to serve my local .m2 directory as static files to emulate a remote server, and Ivy (and likely Maven and Gradle) makes use of HEAD requests.
Ability to set Content-Type of static resource
I had this simple route, serving the content of the assets
resource folder when getting assets/*
static("assets") {
resources("assets")
}
I realized that Ktor was responding to these requests with a weird Content-Type
for files ending in .map
like *.js.map
(or *.css.map
I guess).
This doesn't cause any real issue as most clients getting this kind of files ignore content type entirely. Still, I wanted to change that content type to something a little more standard like application/octet-stream
or even something less standard but more useful for debugging, application/json
.
So I tried to change this content type and couldn't find a way. I tried with an interceptor but you're somehow not allowed to change the content type header.
I ended up having to drop the use of static
/resources
. So now I have this:
val oneYearInSeconds = Duration.ofDays(365).toSeconds()
val pathParameterName = "path"
get("/assets/{$pathParameterName...}") {
val relativePath = call.parameters
.getAll(pathParameterName)
?.joinToString(File.separator)
?: return@get
val content = call.resolveResource(relativePath, "assets") { extension ->
when {
extension.endsWith(".js.map") -> ContentType.Application.Json
else -> ContentType.defaultForFileExtension(extension)
}
} ?: return@get
call.response.header(HttpHeaders.CacheControl, "public, max-age=$oneYearInSeconds, immutable")
call.respond(content)
}
Which is kinda longer and unfortunately doesn't use what's already there for static resources.
(I also implemented caching there, because 1. there does not seem to be caching enabled by default for static content and 2. Ktor's caching headers feature kinda sucks, but that's a different story)
Metrics: ClassCastException when the DropwizardMetrics plugin is installed after the MicrometerMetrics plugin
Using the code found in the micrometer-metrics code snippet, and adding the dependencies for the dropwizard-metrics, if the plugin installation block for the Dropwizard plugin is added after the MicrometerMetrics plugin:
install(MicrometerMetrics) {
registry = PrometheusMeterRegistry(PrometheusConfig.DEFAULT)
distributionStatisticConfig = DistributionStatisticConfig.Builder()
.percentilesHistogram(true)
.maximumExpectedValue(Duration.ofSeconds(20).toNanos().toDouble())
.serviceLevelObjectives(
Duration.ofMillis(100).toNanos().toDouble(),
Duration.ofMillis(500).toNanos().toDouble()
)
.build()
meterBinders = listOf(
ClassLoaderMetrics(),
JvmMemoryMetrics(),
JvmGcMetrics(),
ProcessorMetrics(),
JvmThreadMetrics(),
FileDescriptorMetrics(),
UptimeMetrics()
)
}
install(DropwizardMetrics) {
JmxReporter.forRegistry(registry)
.convertRatesTo(TimeUnit.SECONDS)
.convertDurationsTo(TimeUnit.MILLISECONDS)
.build()
.start()
}
then when the /metrics
endpoint is scrapped, an exception is thrown;
java.lang.ClassCastException: class io.ktor.server.metrics.dropwizard.CallMeasure cannot be cast to class io.ktor.server.metrics.micrometer.CallMeasure (io.ktor.server.metrics.dropwizard.CallMeasure and io.ktor.server.metrics.micrometer.CallMeasure are in unnamed module of loader 'app')
at io.ktor.server.metrics.micrometer.MicrometerMetricsKt$MicrometerMetrics$2$4.invoke(MicrometerMetrics.kt:148)
at io.ktor.server.metrics.micrometer.MicrometerMetricsKt$MicrometerMetrics$2$4.invoke(MicrometerMetrics.kt:146)
at io.ktor.server.application.hooks.ResponseSent$install$1.invokeSuspend(CommonHooks.kt:110)
at io.ktor.server.application.hooks.ResponseSent$install$1.invoke(CommonHooks.kt)
at io.ktor.server.application.hooks.ResponseSent$install$1.invoke(CommonHooks.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(SuspendFunctionGun.kt:88)
at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invokeSuspend(DefaultTransform.kt:29)
at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.engine.DefaultEnginePipelineKt.tryRespondError(DefaultEnginePipeline.kt:122)
at io.ktor.server.engine.DefaultEnginePipelineKt.handleFailure(DefaultEnginePipeline.kt:54)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invokeSuspend(DefaultEnginePipeline.kt:40)
A workaround appears to be registering the Dropwizard plugin before the MicrometerMetrics plugin
Support serving static files from resources in GraalVM native image
It is now not possible to serve static files from resources in GraalVM native image with default configuration:
routing {
static("/assets") {
resources("assets")
}
}
It is because internal fun resourceClasspathResource
supports only file
, jar
and jrt
protocols.
But in native image, resources are supported by resource
protocol.
The default implementation of challengeFunction is empty, causing no session users to access protected resources
Module: ktor-server-auth-jvm-2.2.2.jar
Class: io/ktor/server/auth/SessionAuth.kt:123
This line of code does not work for unauthorized users:
session<UserSession>("auth-session")
Since the default challengeFunction parameter is an empty implementation, users without a Session can also access resources protected by auth-session
{width=70%}
install(Authentication) {
form("auth-form") {
validate { credentials ->
if (credentials.name == "123" && credentials.password == "123") {
UserIdPrincipal(credentials.name)
} else {
null
}
}
}
session<UserSession>("auth-session") // not work
}
Support regex patterns in routing
Wrong ContentType for .mjs files
Steps to reproduce: serve static JS module files from a folder in a standard way like
routing {
trace { application.log.trace(it.buildText()) }
static("/scripts/") {
this.staticRootFolder = File(rootPath)
files(".")
}
}
The problem: the MIME type for .mjs files is wrong and the browser cannot load the modules.
Error in the browser console:
Loading module from “http://localhost:81/scripts/asdf.mjs” was blocked because of a disallowed MIME type (“application/octet-stream”).
Expected result: ContentType header should be "text/javascript"
Allow passing multiple acceptable content types to accept route selector
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1781
Subsystem
Server, route building
Is your feature request related to a problem? Please describe.
Right now the accept()
route selector only takes a single content type.
Describe the solution you'd like
It would be convenient if we could pass more than one, and it could match on any of the ones provided.
Motivation to include to ktor
That would reduce the amount of duplication when multiple content types are handled by the same handler.
Conflict between `ContentNegotiation` and `Mustache` plugins
If I install(ContentNegotiation) { gson() }
and install(Mustache)
call.respond(MustacheContent("temp.hbs", mapOf("user" to sampleUser)))
stops working correctly
Instead of returning proper html output, the server will return a json
{"template":"temp.hbs","model":{"user":{"id":1,"name":"John"}},"contentType":{"contentType":"text","contentSubtype":"html","content":"text/html","parameters":[{"name":"charset","value":"UTF-8","escapeValue":false}]}}
the return content type is application/json
After removing install(ContentNegotiation) { gson() }
block, html is responded correctly but other api routes throw serialisation related errors as expected due to lack of gson
Websockets: connection should be failed immediately when no continuation frame goes after a fragmented text frame
To reproduce send a text Message fragmented into 2 fragments, then Continuation Frame with FIN = false where there is nothing to continue, then an unfragmented Text Message, all sent in one chop.
Expected behavior: the connection is failed immediately since there is no message to continue.
Actual behavior: two text frames are sent back.
embeddedServer(CIO, host = "127.0.0.1", port = 0) {
install(WebSockets)
routing {
webSocket("/") {
for (frame in incoming) {
outgoing.send(frame)
}
}
}
}
Shared
Websockets: Erroneous trace log about expired websocket pings
I have written a simple websocket server with the following configuration:
install(WebSockets) {
pingPeriod = Duration.ofSeconds(30)
}
Here is an excerpt of the logs:
2023-03-19 17:49:25.020 [DefaultDispatcher-worker-25] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: sending ping frame
2023-03-19 17:49:25.023 [DefaultDispatcher-worker-39] TRACE io.ktor.websocket.WebSocket - WebSocketSession(StandaloneCoroutine{Active}@5a55a4c1) receiving frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:49:25.023 [DefaultDispatcher-worker-39] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: received valid pong frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:49:25.023 [DefaultDispatcher-worker-39] TRACE io.ktor.websocket.WebSocket - WebSocket pinger has timed out
2023-03-19 17:49:55.028 [DefaultDispatcher-worker-5] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: sending ping frame
2023-03-19 17:49:55.029 [DefaultDispatcher-worker-52] TRACE io.ktor.websocket.WebSocket - WebSocketSession(StandaloneCoroutine{Active}@5a55a4c1) receiving frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:49:55.029 [DefaultDispatcher-worker-18] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: received valid pong frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:49:55.029 [DefaultDispatcher-worker-18] TRACE io.ktor.websocket.WebSocket - WebSocket pinger has timed out
2023-03-19 17:50:25.035 [DefaultDispatcher-worker-30] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: sending ping frame
2023-03-19 17:50:25.037 [DefaultDispatcher-worker-15] TRACE io.ktor.websocket.WebSocket - WebSocketSession(StandaloneCoroutine{Active}@5a55a4c1) receiving frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:50:25.038 [DefaultDispatcher-worker-15] TRACE io.ktor.websocket.WebSocket - WebSocket Pinger: received valid pong frame Frame PONG (fin=true, buffer len = 76)
2023-03-19 17:50:25.038 [DefaultDispatcher-worker-15] TRACE io.ktor.websocket.WebSocket - WebSocket pinger has timed out
You can see that each time the server sends a PING
frame, it immediately receives a PONG
one, but each time we always get the following log:
TRACE io.ktor.websocket.WebSocket - WebSocket pinger has timed out
This is NOT true, the pong frame has been received in time (which is the default here, 15 seconds).
LOGGER.trace("WebSocket pinger has timed out")
if (rc == null) {
// timeout
// we were unable to send the ping or hadn't got a valid pong message in time,
// so we are triggering close sequence (if already started then the following close frame could be ignored)
onTimeout(CloseReason(CloseReason.Codes.INTERNAL_ERROR, "Ping timeout"))
break
}
Shouldn't we put the log in the condition body? Currently, this is confusing.
Test Infrastructure
runBlocking in TestApplicationEngine loses coroutineContext
TestApplicationEngine
has a few runBlocking
calls inside it.
This can cause trouble for advanced testing, when a suspend function is tested with a coroutineContext
.
One notable example is Kotest. In order to use withClue
, coroutineContext
should contain errorCollectorContextElement
. Otherwise context switching might result in exception. Right now there is no way to pass it into internal runBlocking
calls of TestApplicationEngine
, and it isn't passed from parent context.
Here's a sample code which reproduces the situation above:
@Test
fun test() = runBlocking(errorCollectorContextElement) {
testApplication {
withContext(Dispatchers.Unconfined) {
withClue("some hint") {
withContext(Dispatchers.Unconfined) {
delay(2)
1 shouldBe 1
}
}
}
}
}
Kotest throws an IndexOutOfBoundsException
here, because an error hint is popped from wrong errorCollector (one from another thread).
This particular error can be fixed on Kotest side, but I assume where might be more situations similar to this. Another example might be Exposed framework. One can create a wrapper transaction to rollback it after the test is finished, and context switching might result in losing context with the wrapper transaction.
Other
Missing ktor plugin usages for versions 231.*
Simplify Static Content Plugin
Design defaults for the static content plugin to make it work out of the box.
Upgrade Client Apache Engine Version to use Apache 5
Update reported dependencies
Please use separate commit for every dependency
Requests don't match in nested Regex Routing
routing {
route(Regex("""/abc/""")) {
get(Regex("""cde""")) {
call.respondText { "OK" }
}
}
}
None of the /abc/cde
and /abc//cde
is matched
IllegalArgumentException in Regex Routing
routing {
get(Regex("""\(?<a>\)""")) {
call.respondText { "OK" }
}
}
On request GET /<a>)
there is IllegalArgumentException: No group with name <a>
Unneeded escaping in Regex Routing isn't processed
get(Regex("""\/abc"""))
should have the same behavior as get(Regex("""/abc"""))
Reading response of HEAD request breaks when Content-Length > 0
Add opportunity to pass type info into WebSockets serializing methods
Update JTE to 2.3.0
Update Kotlin to 1.8.10
AutoHead should dispose response body
Comparable HttpStatusCode
Subsystem
Client/Server
Motivation
It is much more useful to compare status codes between each other. For example:
httpResponse.status < HttpStatusCode.OK
Currently, in case, if I want to compare status codes I should use next comparison:
httpResponse.status.value < HttpStatusCode.OK.value
Solution
In my PR HttpStatusCode
will implements Comparable
interface and realize compareTo
with any other HttpStatusCode
Add shutdown configuration for engine in stop method
There are two parameters that are used for shutdown configuration: gracePeriod
and shutdownTimeout
`. There are used in ShutdownHook and stop method
In KTOR-5359 configuration for ShutdownHook
was added
Incorrect handling of private cache directive in HttpCachePlugin
Multiple problems related to private responses caching:
- private cache responses can be shared between multiple users when using HttpClient in a proxy use case
s-maxage
had a typo and was used for private responses, while should be used for shared- only first
Cache-Control
header was checked, but they should be merged