Changelog 2.0 version
2.0.3
released 28th June 2022
Client
Unable to set the Content-Type header in a request
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1127
Ktor Version
1.1.4
Ktor Engine Used(client or server and name)
Apache
JVM Version, Operating System and Relevant Context
1.8, macOS Mojave and Linux Mint, IDEA 2019.1.2 CE
Feedback
I'm accessing and API that requires me to set Content-Type
for all requests, but I'm getting io.ktor.http.UnsafeHeaderException: Header Content-Type is controlled by the engine and cannot be set explicitly
Current implementation:
val client = HttpClient(Apache) {
install(JsonFeature) {
serializer = GsonSerializer()
}
install(Logging) {
level = LogLevel.HEADERS
}
defaultRequest {
header("Content-Type", "application/vnd.api+json")
}
}
get("/test") {
client.get<String>("url...") {
headers {
// other headers
}
}
}
submitFormWithBinaryData call leads to NPE when the ContentNegotiation plugin is installed
Ktor: 2.0.1
Tested on: iPhone 7 (15.1)
Code example (can try to do mini sample if needed):
val client = HttpClient {
install(ContentNegotiation) {
json()
}
defaultRequest {
headers.appendIfNameAbsent(HttpHeaders.ContentType, ContentType.Application.Json.toString())
}
}
client.submitFormWithBinaryData(url, formData {
append("content", bytes, Headers.build {
append(HttpHeaders.ContentType, ContentType.Application.OctetStream)
append(HttpHeaders.ContentDisposition, "filename=content")
})
}).body()
Crash log:
Uncaught Kotlin exception: kotlin.NullPointerException
at 0 MyApp 0x10457e7f7 kfun:kotlin.Throwable#<init>(){} + 75
at 1 MyApp 0x104577147 kfun:kotlin.Exception#<init>(){} + 67
at 2 MyApp 0x10457739b kfun:kotlin.RuntimeException#<init>(){} + 67
at 3 MyApp 0x104577f73 kfun:kotlin.NullPointerException#<init>(){} + 67
at 4 MyApp 0x1045aa93b ThrowNullPointerException + 107
at 5 MyApp 0x10494b6db kfun:io.ktor.client.plugins.contentnegotiation.ContentNegotiation.Plugin.$install$lambda-1COROUTINE$136.invokeSuspend#internal + 4559
at 6 MyApp 0x10494c717 kfun:io.ktor.client.plugins.contentnegotiation.ContentNegotiation.Plugin.$install$lambda-1COROUTINE$136.invoke#internal + 275
at 7 MyApp 0x10494c82f kfun:io.ktor.client.plugins.contentnegotiation.ContentNegotiation.Plugin.$install$lambda-1COROUTINE$136.invoke#internal.223 + 211
at 8 MyApp 0x104809e3b kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 787
at 9 MyApp 0x1048096cf kfun:io.ktor.util.pipeline.SuspendFunctionGun#proceed(){}1:0 + 347
at 10 MyApp 0x10480988b kfun:io.ktor.util.pipeline.SuspendFunctionGun#proceedWith(1:0){}1:0 + 175
at 11 MyApp 0x1049138df kfun:io.ktor.client.plugins.HttpCallValidator.Companion.$install$lambda-1COROUTINE$155.invokeSuspend#internal + 883
at 12 MyApp 0x104914147 kfun:io.ktor.client.plugins.HttpCallValidator.Companion.$install$lambda-1COROUTINE$155.invoke#internal + 275
at 13 MyApp 0x10491425f kfun:io.ktor.client.plugins.HttpCallValidator.Companion.$install$lambda-1COROUTINE$155.invoke#internal.183 + 211
at 14 MyApp 0x104809e3b kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 787
at 15 MyApp 0x1048096cf kfun:io.ktor.util.pipeline.SuspendFunctionGun#proceed(){}1:0 + 347
at 16 MyApp 0x10491e9ab kfun:io.ktor.client.plugins.HttpRequestLifecycle.Plugin.$install$lambda-0COROUTINE$164.invokeSuspend#internal + 1039
at 17 MyApp 0x10491f19f kfun:io.ktor.client.plugins.HttpRequestLifecycle.Plugin.$install$lambda-0COROUTINE$164.invoke#internal + 275
at 18 MyApp 0x10491f2b7 kfun:io.ktor.client.plugins.HttpRequestLifecycle.Plugin.$install$lambda-0COROUTINE$164.invoke#internal.197 + 211
at 19 MyApp 0x104809e3b kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 787
at 20 MyApp 0x1048096cf kfun:io.ktor.util.pipeline.SuspendFunctionGun#proceed(){}1:0 + 347
at 21 MyApp 0x104809a53 kfun:io.ktor.util.pipeline.SuspendFunctionGun#execute(1:0){}1:0 + 387
at 22 MyApp 0x104802453 kfun:io.ktor.util.pipeline.Pipeline#execute(1:1;1:0){}1:0 + 379
at 23 MyApp 0x1048ecb03 kfun:io.ktor.client.HttpClient.$executeCOROUTINE$138#invokeSuspend(kotlin.Result<kotlin.Any?>){}kotlin.Any? + 587
at 24 MyApp 0x1048ecdc7 kfun:io.ktor.client.HttpClient#execute(io.ktor.client.request.HttpRequestBuilder){}io.ktor.client.call.HttpClientCall + 271
at 25 MyApp 0x104942e6b kfun:io.ktor.client.statement.HttpStatement.$executeUnsafeCOROUTINE$189#invokeSuspend(kotlin.Result<kotlin.Any?>){}kotlin.Any? + 743
at 26 MyApp 0x1049431cf kfun:io.ktor.client.statement.HttpStatement#executeUnsafe(){}io.ktor.client.statement.HttpResponse + 239
at 27 MyApp 0x104941f33 kfun:io.ktor.client.statement.HttpStatement.$executeCOROUTINE$186#invokeSuspend(kotlin.Result<kotlin.Any?>){}kotlin.Any? + 811
at 28 MyApp 0x10494297b kfun:io.ktor.client.statement.HttpStatement#execute(kotlin.coroutines.SuspendFunction1<io.ktor.client.statement.HttpResponse,0:0>){0§<kotlin.Any?>}0:0 + 271
at 29 MyApp 0x104942a77 kfun:io.ktor.client.statement.HttpStatement#execute(){}io.ktor.client.statement.HttpResponse + 183
at 30 MyApp 0x104ba3f6b kfun:com.mjegorovas.MyApp.server.MyAppApi.$storeFileContentCOROUTINE$58#invokeSuspend(kotlin.Result<kotlin.Any?>){}kotlin.Any? + 1855
at 31 MyApp 0x104ba473f kfun:com.mjegorovas.MyApp.server.MyAppApi#storeFileContent(kotlin.ByteArray;kotlin.Int;kotlin.Long;kotlin.Int?){} + 367
at 32 MyApp 0x104ba4897 kfun:com.mjegorovas.MyApp.server.MyAppApi#storeFileContent$default(kotlin.ByteArray;kotlin.Int;kotlin.Long;kotlin.Int?;kotlin.Int){} + 283
at 33 MyApp 0x104c7016f kfun:com.mjegorovas.MyApp.useCases.UploadItemUseCase.$storeItems$<anonymous>_3COROUTINE$101.invokeSuspend#internal + 803
at 34 MyApp 0x104c706cf kfun:com.mjegorovas.MyApp.useCases.UploadItemUseCase.$storeItems$<anonymous>_3COROUTINE$101.invoke#internal + 255
at 35 MyApp 0x104c70907 kfun:com.mjegorovas.MyApp.useCases.UploadItemUseCase.$storeItems$<anonymous>_3COROUTINE$101.$<bridge-NNNNB>invoke(kotlin.ByteArray;kotlin.Long){}kotlin.Any?#internal + 227
at 36 MyApp 0x104c78fa3 kfun:com.mjegorovas.MyApp.util.$readByChunksCOROUTINE$107#invokeSuspend(kotlin.Result<kotlin.Any?>){}kotlin.Any? + 1039
at 37 MyApp 0x104c7935f kfun:com.mjegorovas.MyApp.util#readByChunks__at__okio.BufferedSource(kotlin.Long;kotlin.coroutines.SuspendFunction2<kotlin.ByteArray,kotlin.Long,kotlin.Unit>){} + 295
at 38 MyApp 0x104c7947f kfun:com.mjegorovas.MyApp.util#readByChunks$default__at__okio.BufferedSource(kotlin.Long;kotlin.coroutines.SuspendFunction2<kotlin.ByteArray,kotlin.Long,kotlin.Unit>;kotlin.Int){} + 231
at 39 MyApp 0x104c6eee7 kfun:com.mjegorovas.MyApp.useCases.UploadItemUseCase.$storeItemsCOROUTINE$100.invokeSuspend#internal + 2483
at 40 MyApp 0x104c6f873 kfun:com.mjegorovas.MyApp.useCases.UploadItemUseCase.storeItems#internal + 319
at 41 MyApp 0x104c6d8a7 kfun:com.mjegorovas.MyApp.useCases.UploadItemUseCase.$uploadCOROUTINE$99#invokeSuspend(kotlin.Result<kotlin.Any?>){}kotlin.Any? + 3335
at 42 MyApp 0x104582943 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 563
at 43 MyApp 0x1047019fb kfun:kotlinx.coroutines#resume__at__kotlinx.coroutines.DispatchedTask<0:0>(kotlin.coroutines.Continuation<0:0>;kotlin.Boolean){0§<kotlin.Any?>} + 1059
at 44 MyApp 0x104701213 kfun:kotlinx.coroutines#dispatch__at__kotlinx.coroutines.DispatchedTask<0:0>(kotlin.Int){0§<kotlin.Any?>} + 839
at 45 MyApp 0x1046afa07 kfun:kotlinx.coroutines.CancellableContinuationImpl.dispatchResume#internal + 115
at 46 MyApp 0x1046b00e7 kfun:kotlinx.coroutines.CancellableContinuationImpl.resumeImpl#internal + 783
at 47 MyApp 0x1046b0337 kfun:kotlinx.coroutines.CancellableContinuationImpl#resumeImpl$default(kotlin.Any?;kotlin.Int;kotlin.Function1<kotlin.Throwable,kotlin.Unit>?;kotlin.Int){} + 243
at 48 MyApp 0x1046aebcb kfun:kotlinx.coroutines.CancellableContinuationImpl#resumeWith(kotlin.Result<1:0>){} + 207
at 49 MyApp 0x1046d4767 kfun:kotlinx.coroutines.ResumeOnCompletion.invoke#internal + 383
at 50 MyApp 0x1046c5c33 kfun:kotlinx.coroutines.JobSupport.notifyCompletion#internal + 703
at 51 MyApp 0x1046c4e9b kfun:kotlinx.coroutines.JobSupport.completeStateFinalization#internal + 1167
at 52 MyApp 0x1046c34bb kfun:kotlinx.coroutines.JobSupport.finalizeFinishingState#internal + 1131
at 53 MyApp 0x1046cd387 kfun:kotlinx.coroutines.JobSupport.continueCompleting#internal + 371
at 54 MyApp 0x1046d030b kfun:kotlinx.coroutines.JobSupport.ChildCompletion.invoke#internal + 267
at 55 MyApp 0x1046c5c33 kfun:kotlinx.coroutines.JobSupport.notifyCompletion#internal + 703
at 56 MyApp 0x1046c4e9b kfun:kotlinx.coroutines.JobSupport.completeStateFinalization#internal + 1167
at 57 MyApp 0x1046c34bb kfun:kotlinx.coroutines.JobSupport.finalizeFinishingState#internal + 1131
at 58 MyApp 0x1046cc68b kfun:kotlinx.coroutines.JobSupport.tryMakeCompletingSlowPath#internal + 1371
at 59 MyApp 0x1046cc063 kfun:kotlinx.coroutines.JobSupport.tryMakeCompleting#internal + 639
at 60 MyApp 0x1046cbbf7 kfun:kotlinx.coroutines.JobSupport#makeCompletingOnce(kotlin.Any?){}kotlin.Any? + 355
at 61 MyApp 0x1046a4797 kfun:kotlinx.coroutines.AbstractCoroutine#resumeWith(kotlin.Result<1:0>){} + 219
at 62 MyApp 0x104582c23 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 1299
at 63 MyApp 0x10470050b kfun:kotlinx.coroutines.DispatchedTask#run(){} + 2743
at 64 MyApp 0x1046b982f kfun:kotlinx.coroutines.EventLoop#processUnconfinedEvent(){}kotlin.Boolean + 351
at 65 MyApp 0x104701463 kfun:kotlinx.coroutines.resumeUnconfined#internal + 439
at 66 MyApp 0x1047011af kfun:kotlinx.coroutines#dispatch__at__kotlinx.coroutines.DispatchedTask<0:0>(kotlin.Int){0§<kotlin.Any?>} + 739
at 67 MyApp 0x1046afa07 kfun:kotlinx.coroutines.CancellableContinuationImpl.dispatchResume#internal + 115
at 68 MyApp 0x1046b0f6f kfun:kotlinx.coroutines.CancellableContinuationImpl#completeResume(kotlin.Any){} + 115
at 69 MyApp 0x1046de31f kfun:kotlinx.coroutines.channels.AbstractChannel.ReceiveHasNext.resumeReceiveClosed#internal + 499
at 70 MyApp 0x1046e45bb kfun:kotlinx.coroutines.channels.AbstractSendChannel.helpClose#internal + 979
at 71 MyApp 0x1046e37a3 kfun:kotlinx.coroutines.channels.AbstractSendChannel#close(kotlin.Throwable?){}kotlin.Boolean + 591
at 72 MyApp 0x1046f3b4f kfun:kotlinx.coroutines.channels.SendChannel#close$default(kotlin.Throwable?;kotlin.Int){}kotlin.Boolean + 263
at 73 MyApp 0x104966f6b kfun:io.ktor.client.engine.darwin.DarwinResponseReader#objc:URLSession:task:didCompleteWithError: + 451
at 74 MyApp 0x104969963 _696f2e6b746f723a6b746f722d636c69656e742d64617277696e_knbridge1109 + 299
at 75 CFNetwork 0x1840bc4ff CFURLRequestCopyHTTPRequestMethod + 2203
at 76 libdispatch.dylib 0x183629193 <redacted> + 23
at 77 libdispatch.dylib 0x18362a197 <redacted> + 15
at 78 libdispatch.dylib 0x1835d65cf <redacted> + 939
at 79 CoreFoundation 0x1839197e7 <redacted> + 11
at 80 CoreFoundation 0x1838d70e7 <redacted> + 2527
at 81 CoreFoundation 0x1838e9d7b CFRunLoopRunSpecific + 571
at 82 GraphicsServices 0x19db5e99f GSEventRunModal + 159
at 83 UIKitCore 0x18611c05b <redacted> + 1079
at 84 UIKitCore 0x185eb1cdf UIApplicationMain + 2027
at 85 SwiftUI 0x18af821cb <redacted> + 159
at 86 SwiftUI 0x18aecbdd7 <redacted> + 179
at 87 SwiftUI 0x18aeb15ff $s7SwiftUI3AppPAAE4mainyyFZ + 95
at 88 MyApp 0x104502b5b $s5MyApp6iOSAppV5$mainyyFZ + 39
at 89 MyApp 0x104502c07 main + 11
at 90 dyld 0x1064a018f 0x0 + 4400480655
Signal: SIGABRT (signal SIGABRT)
Setting body to TextContent leads to NPE when the ContentNegotiation plugin is installed
When trying to send a String containing json with ktor while also having serializers installed I noticed that it was breaking my json string.
I tried bypassing the serializers by using TextContent because this doesn't set the bodyType on the HttpRequestBuilder.
This resulted in the following stacktrace:
java.lang.NullPointerException: null
at io.ktor.client.plugins.contentnegotiation.ContentNegotiation$Plugin$install$1.invokeSuspend(ContentNegotiation.kt:112)
at io.ktor.client.plugins.contentnegotiation.ContentNegotiation$Plugin$install$1.invoke(ContentNegotiation.kt)
at io.ktor.client.plugins.contentnegotiation.ContentNegotiation$Plugin$install$1.invoke(ContentNegotiation.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:123)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:81)
at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(SuspendFunctionGun.kt:91)
at io.ktor.client.plugins.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:125)
at io.ktor.client.plugins.HttpCallValidator$Companion$install$1.invoke(HttpCallValidator.kt)
at io.ktor.client.plugins.HttpCallValidator$Companion$install$1.invoke(HttpCallValidator.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:123)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:81)
at io.ktor.client.plugins.HttpRequestLifecycle$Plugin$install$1.invokeSuspend(HttpRequestLifecycle.kt:35)
at io.ktor.client.plugins.HttpRequestLifecycle$Plugin$install$1.invoke(HttpRequestLifecycle.kt)
at io.ktor.client.plugins.HttpRequestLifecycle$Plugin$install$1.invoke(HttpRequestLifecycle.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:123)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:81)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:101)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.client.HttpClient.execute$ktor_client_core(HttpClient.kt:184)
at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:107)
at io.ktor.client.statement.HttpStatement.execute(HttpStatement.kt:46)
at io.ktor.client.statement.HttpStatement.execute(HttpStatement.kt:61)
at me.melijn.bot.commands.DevExtension$setup$2$1.invokeSuspend(DevExtension.kt:152)
at me.melijn.bot.commands.DevExtension$setup$2$1.invoke(DevExtension.kt)
at me.melijn.bot.commands.DevExtension$setup$2$1.invoke(DevExtension.kt)
at com.kotlindiscord.kord.extensions.commands.chat.ChatCommand$call$2.invokeSuspend(ChatCommand.kt:434)
at com.kotlindiscord.kord.extensions.commands.chat.ChatCommand$call$2.invoke(ChatCommand.kt)
at com.kotlindiscord.kord.extensions.commands.chat.ChatCommand$call$2.invoke(ChatCommand.kt)
at com.kotlindiscord.kord.extensions.types.Lockable$DefaultImpls.withLock(Lockable.kt:24)
at com.kotlindiscord.kord.extensions.commands.Command.withLock(Command.kt:39)
at com.kotlindiscord.kord.extensions.commands.chat.ChatCommand.call$suspendImpl(ChatCommand.kt:351)
at com.kotlindiscord.kord.extensions.commands.chat.ChatCommand.call(ChatCommand.kt)
at com.kotlindiscord.kord.extensions.commands.chat.ChatCommand.call$default(ChatCommand.kt:345)
at com.kotlindiscord.kord.extensions.commands.chat.ChatCommandRegistry.handleEvent$suspendImpl(ChatCommandRegistry.kt:237)
at com.kotlindiscord.kord.extensions.commands.chat.ChatCommandRegistry$handleEvent$1.invokeSuspend(ChatCommandRegistry.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:749)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Here is a code snippet to reproduce this:
I've had it with this issue editor, it wants to place everything on one line so here it is ig..
https://gist.github.com/ToxicMushroom/975601e26be80994a57487e043a3e6dc
[JS client] Cannot change redirect policy by followRedirects=false
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1968
Ktor Version and Engine Used (client or server and name)
Ktor HTTP client 1.3.2 on Kotlin/JS; dependencies:
implementation("io.ktor:ktor-client-json:$ktorVersion")
implementation("io.ktor:ktor-client-js:$ktorVersion")
implementation("io.ktor:ktor-client-json-js:$ktorVersion")
implementation("io.ktor:ktor-client-serialization-js:$ktorVersion")
Describe the bug
While debugging the situation with redirects location, I found that if you call something like println(resp)
where resp is a HttpResponse
result of a request that returns HTTP/1.1 302 Found
, string will be HttpResponse[http://localhost:3000/api/v1/auth/logout, 200 OK]
, where I excected something like HttpResponse[http://localhost:3000/api/v1/auth/logout, 302 Found]
.
In network tab of browser developer tools response shown as 302 Found
with correct Location
header, but ktor client shows different results...
To Reproduce
Steps to reproduce the behavior:
- Take this repo for example, change url in
App.kt
to something suitable in your environment (httpbin, for example) - Open in browser localhost:3000
- See string representation of response with 200 OK
Expected behavior
302 Found with Location
header
Screenshots
If what, I can try to debug this and fix by myself, but later...
P.S. Thank you for ktor framework and client. They're awesome.
WebSocket client closes connection due to an HTTP request timeout
This broke between 2.0.1 and 2.0.2.
val websocketClient = HttpClient(CIO) {
install(HttpTimeout) {
connectTimeoutMillis = 2000L
}
install(WebSockets) {
pingInterval = 30000L
}
}
val session = websocketClient.webSocketSession("ws://some-url")
session.launch {
while (isActive) {
val frame = session.incoming.receive()
// to things with frame
}
}
The websocket server shows the client as connected and eventually reports an abnormal close. On the client side the coroutine is cancelled with a request timeout exception from ktor:
java.util.concurrent.CancellationException: Request is timed out
at kotlinx.coroutines.ExceptionsKt.CancellationException(Exceptions.kt:22)
at kotlinx.coroutines.JobKt__JobKt.cancel(Job.kt:596)
at kotlinx.coroutines.JobKt.cancel(Unknown Source)
at io.ktor.client.engine.cio.EndpointKt$setupTimeout$timeoutJob$1.invokeSuspend(Endpoint.kt:250)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:749)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: io.ktor.client.plugins.HttpRequestTimeoutException: Request timeout has expired [url=ws://mrf-rest.lab2018.qa.lbaum.eu:8081/v2/events/3853126682810736, request_timeout=unknown ms]
... 7 common frames omitted
CIO: Websockets request doesn't include query parameters
The Websocket HttpClient doesn't seem to send the query parameters since Ktor 2.0+. It looks like they are stripped.
Ktor Client code:
val client = HttpClient(CIO) {
install(WebSockets)
}
client.wss(
host = "api.acme.com",
path = "/connect?param1=value1¶m2=value2"
) {
...
}
Ktor Server code:
webSocket("/connect") {
logger.debug("call.request.path(): ${call.request.path()}")
logger.debug("call.request.uri: ${call.request.uri}")
logger.debug("call.request.rawQueryParameters: ${call.request.rawQueryParameters}")
logger.debug("call.request.queryParameters: ${call.request.queryParameters}")
logger.debug("call.parameters: ${call.parameters}")
logger.debug("call.request.queryParameters[\"param1\"]: ${call.request.queryParameters["param1"]}")
logger.debug("call.request.queryParameters[\"param2\"]: ${call.request.queryParameters["param2"]}")
}
Using the client code above with Ktor 1.6.7 results on the server:
call.request.path(): /connect
call.request.uri: /connect?param1=value1¶m2=value2
call.request.rawQueryParameters: Parameters [param1=[value1], param2=[value2]]
call.request.queryParameters: io.ktor.server.netty.NettyApplicationRequest$queryParameters$1@3456f3cc
call.parameters: Parameters [param1=[value1], param2=[value2]]
call.request.queryParameters["param1"]: value1
call.request.queryParameters["param2"]: value2
Using the client code above with Ktor 2.0.1 results on the server:
call.request.path(): /connect
call.request.uri: /connect
call.request.rawQueryParameters: Parameters []
call.request.queryParameters: io.ktor.server.netty.NettyApplicationRequest$queryParameters$1@60a44ee9
call.parameters: Parameters []
call.request.queryParameters["param1"]: null
call.request.queryParameters["param2"]: null
Any help would be much appreciated! :D
Kind regards,
Thomas
Invalid request line produced by CIO engine for URL with parameters and without path
When making a request with URL http://localhost?foo=bar
, CIO engine sends the following data to the server:
GET ?foo=bar/ HTTP/1.1
...
It seems to be caused by this normalization. The normalization should take into account the rest of the URL (parameters, fragment etc.).
Core
UDP responses are received with a huge delay on JVM Windows (due to reverse DNS lookup internally)
The raw UDP sockets in Ktor do not work as expected for me. Requests are OK, but responses come in very slowly, and only gets worse with more responses. In Wireshark you can see responses come in immediately.
After some debugging I found the cause. Apparently every UDP response causes a reverse DNS lookup in Ktor to get the hostname. Call is here (hostName):
This function is used here every time a UDP packet is received:
- https://github.com/ktorio/ktor/blob/3a0accc88eb166a0fc11324f3228d55c2abb845f/ktor-network/jvm/src/io/ktor/network/sockets/DatagramSocketImpl.kt#L81
- https://github.com/ktorio/ktor/blob/3a0accc88eb166a0fc11324f3228d55c2abb845f/ktor-network/jvm/src/io/ktor/network/sockets/DatagramSocketImpl.kt#L99
It does not seem to happen for every device on my local network. But for some devices it causes around a 4 second delay on my Windows PC, measured like the following:
val address = java.net.InetSocketAddress("192.168.86.54", 56700)
measureTimeMillis {
address.hostName
}.also { println(it.toString()) } // prints around 4000 milliseconds
Now comes the worst part. This delay is there every time. So say for example you receive 100 UDP packets at the same time. That means that with Ktor the last packet is received after 100*4 = 400 seconds
This makes the Ktor UDP sockets unusable on Windows, so this needs to be fixed. For some reason this does not happen for every local IP address, and I can also not reproduce it on macOS. It is a Windows only issue for me.
See documentation for InetSocketAddress.html#getHostName()
:
Gets the hostname. Note: This method may trigger a name service reverse lookup if the address was created with a literal IP address.
https://docs.oracle.com/javase/7/docs/api/java/net/InetSocketAddress.html#getHostName()
As an alternative, instead of using getHostName
(hostName
in Kotlin), another function called getHostString
could be used instead. This function specifically mentions in the documentation:
Returns the hostname, or the String form of the address if it doesn't have a hostname (it was created using a literal). This has the benefit of not attempting a reverse lookup.
I think this is a good replacement, although it could be seen as a minor breaking chance in Ktor, but it is much better than a 4 second delay. And there needs to be a fallback for < 1.7 as this function is only available since 1.7.
Some related information:
Thanks for looking into this.
UrlBuilder escapes fragment parameters
val url = URLBuilder(
Url("https://localhost")
).apply {
fragment = "action_uri=%3A"
}.build()
url.toString() // => https://localhost#action_uri=%253A
Url(url.toString()).toString() // => https://localhost#action_uri=%25253A
As you can see, escape sequences in the fragment are parsed without unescaping, but when converting to a string they get escaped.
Docs
Website / docs: Builtin search does not work
{width=70%}
{width=70%}
Clarify that the 'install' function should be called in the client context
ActorSelectorManager should be SelectorManager for a Socket docs
This page is written for Jvm only sockets, but can be tuned for multiplatform:
Generator
Generator generates WebSockets sample with an empty path (`/`) and it clashes with Routing feature's default path
After creating a project with WebSockets feature, you have in Sockets.kt
:
fun Application.configureSockets() {
install(WebSockets) {
...
}
routing {
webSocket("/") { // websocketSession
...
}
}
}
And Routing.kt
:
fun Application.configureRouting() {
routing {
get("/") {
...
}
}
}
WebSockets feature creates non-compilable code with non-exhaustive `when`
Create a project with WebSockets feature and in Sockets.kt
will be:
fun Application.configureSockets() {
install(WebSockets) {
...
}
routing {
webSocket("/") { // websocketSession
for (frame in incoming) {
when (frame) { // COMPILE ERROR: NON-EXHAUSTIVE WHEN
is Frame.Text -> {
...
}
}
}
}
}
}
IntelliJ IDEA Plugin
The 'Create Ktor test' intention should initialize a client with the WebSockets plugin installed
Currently, the plugin adds the following code to a test:
client.webSocket("/chat") {
TODO("Please write your test here")
}
But it would be convenient to initialize a client with the WebSockets plugin installed:
val client = createClient {
install(WebSockets)
}
client.webSocket("/chat") {
TODO("Please write your test here")
}
Migration from 2.0.0 -> 2.0.1 is not Popping Up on ktor-samples repo
The plugin doesn't add a template file when generating a project with Freemarker/etc
To reproduce the issue, generate a new project with the Freemarker plugin and leave the Add sample code
option enabled. The following endpoint will be generated:
get("/html-freemarker") {
call.respond(FreeMarkerContent("index.ftl", mapOf("data" to IndexData(listOf(1, 2, 3))), ""))
}
But the index.ftl
is missing in a project, so accessing this endpoint returns the 500 error:
Exception class freemarker.template.TemplateNotFoundException: Template not found for name "index.ftl".
Help example: https://github.com/ktorio/ktor-documentation/tree/main/codeSnippets/snippets/freemarker
The 'Migrate Ktor to latest version' intention should show some message when a project already on a latest version
Migration action preview should not consume CPU&network before appearing in actions menu
Currently migration action is only shown in Actions menu if any ktor migration is available. In order to decide upon that it does a network request to the generator backend.
Server
Development mode class loader leads to ClassCastException within a CouroutineScope
In a CoroutineScope of a Ktor-2.0.0-application (netty-engine), the "ClassLoaders@AppClassLoader" is not aware of Objects loaded by the (default?) Application ClassLoader. Therefore, for exampe, received objects from an (Apache-ActiveMQ-Artemis-)message-queue cannot be cast to its corresponding DTOs - see attached screenshots with Ktor-1.6.8/Kotlin-1.6.10 vs. Ktor-2.0.0/Kotlin-1.6.20.
call.receiveText() tries to parse body as JSON when the ContentNegotiation plugin is installed
The receiveText() function previously read the body as-is without any additional steps. Starting with version 2.0.2 Ktor tries to parse it as JSON when the request sets a Content-Type header. This is not expected behavior.
In our case we use the receiveText() function to receive JSON bodies in our webhook routes and send the body to third party libraries that do the actual deserialization.
The following code can be used to reproduce the issue (using ktor 2.0.2 and kotlinx.serialization 1.3.3)
embeddedServer(Netty, port = 8080) {
install(ContentNegotiation) {
serialization(ContentType.Application.Json, Json.Default)
}
routing {
post("test") {
val body = call.receiveText()
call.respond("OK")
}
}
}.start(true)
Sending a JSON request responds with "OK" in 2.0.0 and throws an exception in 2.0.2 using the following request.
POST http://localhost:8080/test
Content-Type: application/json
{
"test": true
}
Validate that the body of an incoming request is received completely
From time to time, we receive server requests with an incomplete body.
It seems that the cause of that is lack of `Content-Length` validation during request's body read.
In the `ServletReader` validated excess length of the body, but no assertion is made that received bodySize is equal to the `Content-Length`.
Direct byte buffers are increased in size when server slowly processes request
Hello everyone
We use ktor 2.0.0 with Netty engine
We process large tsv, tsv transmitted over http
We represent http request body as Flow<ByteBuffer>
How we read from channel
fun ApplicationCall.receiveFlow(): Flow<ByteBuffer> = flow<ByteBuffer> {
val channel = receiveChannel()
try {
while (true) {
val buffer = ByteBuffer.allocate(4096)
if (channel.readAvailable(buffer) == -1) {
break
}
buffer.flip()
emit(buffer)
}
} catch (e: Exception) {
log.error("URL: '${request.uri}' error reading body", e)
throw e
}
}
If server slowly process http request - we see increase direct byte buffers (non-heap), this buffers create netty for buffering request (buffers grows to 2 gb)
I expected back pressure in ktor, but buffers are growing.
I communicated with netty users - they recommend disable autoread at channel and use read() manually
Does ktor support back pressure? If so, how enable it?
ResponseConverter NPE when returning ByteArray with the ContentNegotiation plugin
it.converter.serialize(
contentType = contentType ?: it.contentType,
charset = acceptCharset ?: Charsets.UTF_8,
typeInfo = call.response.responseType!!, // NPE triggered by responseType
value = subject
)
Stacktrace:
java.lang.NullPointerException
at io.ktor.server.plugins.contentnegotiation.ResponseConverterKt$convertResponseBody$1$1.invokeSuspend(ResponseConverter.kt:52)
at io.ktor.server.plugins.contentnegotiation.ResponseConverterKt$convertResponseBody$1$1.invoke(ResponseConverter.kt)
at io.ktor.server.plugins.contentnegotiation.ResponseConverterKt$convertResponseBody$1$1.invoke(ResponseConverter.kt)
at io.ktor.server.application.OnCallRespondContext.transformBody(KtorCallContexts.kt:86)
at io.ktor.server.plugins.contentnegotiation.ResponseConverterKt$convertResponseBody$1.invokeSuspend(ResponseConverter.kt:23)
at io.ktor.server.plugins.contentnegotiation.ResponseConverterKt$convertResponseBody$1.invoke(ResponseConverter.kt)
at io.ktor.server.plugins.contentnegotiation.ResponseConverterKt$convertResponseBody$1.invoke(ResponseConverter.kt)
at io.ktor.server.application.PluginBuilder$onDefaultPhase$1.invokeSuspend(PluginBuilder.kt:217)
at io.ktor.server.application.PluginBuilder$onDefaultPhase$1.invoke(PluginBuilder.kt)
at io.ktor.server.application.PluginBuilder$onDefaultPhase$1.invoke(PluginBuilder.kt)
at io.ktor.server.application.PluginBuilder$onDefaultPhaseWithMessage$1$1$1.invokeSuspend(PluginBuilder.kt:200)
at io.ktor.server.application.PluginBuilder$onDefaultPhaseWithMessage$1$1$1.invoke(PluginBuilder.kt)
at io.ktor.server.application.PluginBuilder$onDefaultPhaseWithMessage$1$1$1.invoke(PluginBuilder.kt)
at io.ktor.util.debug.ContextUtilsKt.addToContextInDebugMode(ContextUtils.kt:30)
at io.ktor.server.application.PluginBuilder$onDefaultPhaseWithMessage$1$1.invokeSuspend(PluginBuilder.kt:196)
at io.ktor.server.application.PluginBuilder$onDefaultPhaseWithMessage$1$1.invoke(PluginBuilder.kt)
at io.ktor.server.application.PluginBuilder$onDefaultPhaseWithMessage$1$1.invoke(PluginBuilder.kt)
at io.ktor.server.application.ApplicationPluginKt$addAllInterceptors$1$1$1.invokeSuspend(ApplicationPlugin.kt:167)
at io.ktor.server.application.ApplicationPluginKt$addAllInterceptors$1$1$1.invoke(ApplicationPlugin.kt)
at io.ktor.server.application.ApplicationPluginKt$addAllInterceptors$1$1$1.invoke(ApplicationPlugin.kt)
at io.ktor.util.pipeline.DebugPipelineContext.proceedLoop(DebugPipelineContext.kt:80)
at io.ktor.util.pipeline.DebugPipelineContext.proceed(DebugPipelineContext.kt:57)
at io.ktor.util.pipeline.DebugPipelineContext.execute$ktor_utils(DebugPipelineContext.kt:63)
@Test
fun test() {
testApplication {
application {
routing {
install(ContentNegotiation) {
register(ContentType.Application.Json, JacksonConverter())
}
get("/") {
call.respond("test".toByteArray())
}
}
}
client.get("/")
}
}
"No instance for key AttributeKey: ApplicationPluginRegistry" when exception is thrown during the Call phase
Step to reproduce:
- Run this code fragment:
package ru.vs.control.server
import io.ktor.server.application.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
fun main() {
val env = applicationEngineEnvironment {
connector {
host = "0.0.0.0"
port = 8080
}
module {
intercept(ApplicationCallPipeline.Call) {
throw RuntimeException("my-unexpected-exception")
}
}
}
embeddedServer(Netty, env).start(true)
}
- Send any request to server
Expected result:
Stack trace with my exception inside
Actual result:
Internal ktor engine stack tace:
stack trace
java.lang.IllegalStateException: No instance for key AttributeKey: ApplicationPluginRegistry
at io.ktor.util.Attributes$DefaultImpls.get(Attributes.kt:62) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.AttributesJvmBase.get(AttributesJvm.kt:15) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.logging.LoggingKt.getMdcProvider(Logging.kt:36) ~[ktor-server-core-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.engine.DefaultEnginePipelineKt.logError(DefaultEnginePipeline.kt:59) ~[ktor-server-host-common-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.engine.DefaultEnginePipelineKt.handleFailure(DefaultEnginePipeline.kt:53) ~[ktor-server-host-common-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:46) ~[ktor-server-netty-jvm-2.0.2.jar:2.0.2]
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.6.21.jar:1.6.21-release-334(1.6.21)]
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:141) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.SuspendFunctionGun.access$resumeRootWith(SuspendFunctionGun.kt:14) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:58) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46) ~[kotlin-stdlib-1.6.21.jar:1.6.21-release-334(1.6.21)]
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:141) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.SuspendFunctionGun.access$resumeRootWith(SuspendFunctionGun.kt:14) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:58) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46) ~[kotlin-stdlib-1.6.21.jar:1.6.21-release-334(1.6.21)]
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:141) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:126) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:81) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invokeSuspend(BaseApplicationEngine.kt:122) ~[ktor-server-host-common-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invoke(BaseApplicationEngine.kt) ~[ktor-server-host-common-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invoke(BaseApplicationEngine.kt) ~[ktor-server-host-common-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:123) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:81) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:101) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invokeSuspend(Pipeline.kt:478) ~[ktor-server-host-common-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt) ~[ktor-server-host-common-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt) ~[ktor-server-host-common-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invokeSuspend(DefaultEnginePipeline.kt:118) ~[ktor-server-host-common-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invoke(DefaultEnginePipeline.kt) ~[ktor-server-host-common-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invoke(DefaultEnginePipeline.kt) ~[ktor-server-host-common-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:123) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:81) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:101) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1$invokeSuspend$$inlined$execute$1.invokeSuspend(Pipeline.kt:478) ~[ktor-server-netty-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt) ~[ktor-server-netty-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt) ~[ktor-server-netty-jvm-2.0.2.jar:2.0.2]
at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17) ~[ktor-utils-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:119) ~[ktor-server-netty-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt) ~[ktor-server-netty-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt) ~[ktor-server-netty-jvm-2.0.2.jar:2.0.2]
at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55) ~[kotlinx-coroutines-core-jvm-1.6.1.jar:?]
at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:112) ~[kotlinx-coroutines-core-jvm-1.6.1.jar:?]
at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:126) ~[kotlinx-coroutines-core-jvm-1.6.1.jar:?]
at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:56) ~[kotlinx-coroutines-core-jvm-1.6.1.jar:?]
at kotlinx.coroutines.BuildersKt.launch(Unknown Source) ~[kotlinx-coroutines-core-jvm-1.6.1.jar:?]
at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:37) ~[ktor-server-netty-jvm-2.0.2.jar:2.0.2]
at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:29) ~[ktor-server-netty-jvm-2.0.2.jar:2.0.2]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
at io.ktor.server.netty.EventLoopGroupProxy$Companion.create$lambda-1$lambda-0(NettyApplicationEngine.kt:260) ~[ktor-server-netty-jvm-2.0.2.jar:2.0.2]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.77.Final.jar:4.1.77.Final]
at java.lang.Thread.run(Thread.java:829) [?:?]
CallLogging: JVM crashes when jansi checks whether a file descriptor refers to a terminal
стектрейс и инфо по окружению во вложении
Resources plugin fails to process parameters of type UShort
Sample code that crashes:
package com.example
import io.ktor.resources.Resource
import io.ktor.server.application.Application
import io.ktor.server.application.call
import io.ktor.server.application.install
import io.ktor.server.engine.embeddedServer
import io.ktor.server.netty.Netty
import io.ktor.server.resources.Resources
import io.ktor.server.resources.get
import io.ktor.server.response.respondText
import io.ktor.server.routing.get
import io.ktor.server.routing.routing
import kotlinx.serialization.Serializable
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0") {
configureRouting()
}.start(wait = true)
}
fun Application.configureRouting() {
install(Resources)
routing {
get("/") {
call.respondText("Hello World!")
}
get<UShortTest> {
call.respondText("Res: string=${it.testString}, ushort=${it.testUShort}")
}
}
}
@Serializable
@Resource("/test-ushort")
class UShortTest(val testString: String, val testUShort: UShort)
Crash details:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Array index out of range: -3 at kotlinx.serialization.internal.PluginGeneratedSerialDescriptor.getElementDescriptor(PluginGeneratedSerialDescriptor.kt:135) at io.ktor.resources.serialization.ResourcesFormat.collectAllParameters(ResourcesFormat.kt:76) at io.ktor.resources.serialization.ResourcesFormat.collectAllParameters(ResourcesFormat.kt:78) at io.ktor.resources.serialization.ResourcesFormat.encodeToQueryParameters(ResourcesFormat.kt:64) at io.ktor.server.resources.RoutingKt.resource(Routing.kt:186) ...
While the issue affects UShort, ULong etc, value classes like the one below seem to work fine:
@JvmInline
@Serializable
value class TestCustom(val iamstring: String)
Test Infrastructure
Support the HttpTimeout capability in the DelegatingTestClientEngine
The use case is a client makes a request in a route's handler with a timeout.
fun Application.configureRouting(httpClient: HttpClient) {
routing {
get("/test") {
httpClient.get("http://fake.site.io/toto") {
timeout {
requestTimeoutMillis = 100
}
}
call.respondText("Hello World!")
}
}
}
class ApplicationTest {
@Test
fun testRoot() = testApplication {
val myClient = createClient {
install(HttpTimeout)
}
application {
configureRouting(myClient)
}
externalServices {
hosts ("http://fake.site.io") {
routing {
get("/toto") {
call.respond(HttpStatusCode.OK)
}
}
}
}
myClient.get("/test").apply {
assertEquals(HttpStatusCode.OK, status)
}
}
}
https://kotlinlang.slack.com/archives/C0A974TJ9/p1654617495485869
Other
Limit the number of parallel running requests in Netty
Update kotlinx.coroutines to 1.6.2
Ignore SIGPIPE for server sockets
CIO engine doesn't apply a request timeout from the `HttpTimeout` plugin
Ignore ByteReadChannel as receive type in ContentNegotiation
Non-decipherable exception "No result transformation found"
When something is configured incorrectly, Ktor client launches the following exception
No request transformation found:
java.lang.IllegalStateException: No request transformation found: [Lcom.hadihariri.keolink.LoginData;@23d1e5d0
The only insight it provides is a data class. The help for this exception leads to even less information
As a user, I have no idea what I am meant to do to solve the problem. It would be good to provide some guidance.
Resources plugin doesn't respect default values for Enum
2.0.2
released 30th May 2022
Build System Plugin
Apply `application` plugin automatically
Since ShadowJar and GraalVM plugins rely on application
plugin, it's better to apply this plugin automatically when ktor plugin is applied.
`buildFatJar` task needs `mainClassName` to be set even if `mainClass` is set
We can fix it by setting mainClassName
property from application.mainClass
before running shadowJar
Tasks for publishing image create two images instead of one
publishImage
and publishImageToLocalRegistry
are affected
Client
[iOS] Prevent HttpClient from persisting cookies across requests
Hello!
I'm using Ktor in a KMM app, and I am not having any kind of problem in Android, but in iOS it behaves weird. I don't have the HttpCookies plugin installed, because I'm handling and storing the session cookie outside of Ktor.
What happens is that the very first request that is sent to the server (GET /api/session
) contains a Set-Cookie
header in the response, and for this request I don't save the cookie nor do anything with it, the request serves only to check if there is a session based solely on the response of the server. Both in Android and iOS we get the Set-Cookie
header for this request.
The second request (POST /api/session
) is made after the user attempts to log in using their credentials. For this request, we need the response to contain the Set-Cookie
header so we can store the session token it contains and then pass it onto further requests. This happens in Android, but not in iOS. I tried removing the first request, and magically, I get a cookie for this one!
Is the iOS backend automatically storing and passing the cookie without my knowledge or consent? As I said before, I didn't install the HttpCookies plugin.
Darwin engine: Client connection is closed after each request
I am using the iOS HttpClientEngine of Ktor and the connection seems to be closed after each request, even if I do not intentionally close any connection. Keep Alive header is received correctly in the server and in the Android side (using OkHtttp) the socket connection is maintained to be reused in next requests, but in the iOS side the socket connection is closed after each request.
The output from the request of the iOS App in the server (run in a local machine) is the following:
dbug: Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder[26]
Attempting to validate the bound parameter 'loginRequest' of type 'ApiHosteleriaMobile.Entities.requests.LoginRequest' ...
dbug: Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder[27]
Done attempting to validate the bound parameter 'loginRequest' of type 'ApiHosteleriaMobile.Entities.requests.LoginRequest'.
info: ApiHosteleriaMobile.ParameterAdapterActionFilter[0]
[Request][][/api/v1/login/login]
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[1]
Executing action method ApiHosteleriaMobile.Controllers.LoginController.ValidateLogin (ApiHosteleriaMobile) - Validation state: Invalid
fail: HosteleriaApp.api.Repositories.LoginRepository[0]
[validateUser][ILoginRepository] ErrorContent: No se han encontrado ni operadores ni hosteleros Username:sendoa.vaz@gmail.com
fail: BaseLogger[0]
[LoginUseCase][ErrorCode=6000] Username:sendoa.vaz@gmail.com
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2]
Executed action method ApiHosteleriaMobile.Controllers.LoginController.ValidateLogin (ApiHosteleriaMobile), returned result Microsoft.AspNetCore.Mvc.ObjectResult in 9317.4553ms.
dbug: Microsoft.AspNetCore.Mvc.Infrastructure.DefaultOutputFormatterSelector[11]
List of registered output formatters, in the following order: Microsoft.AspNetCore.Mvc.Formatters.HttpNoContentOutputFormatter, Microsoft.AspNetCore.Mvc.Formatters.StringOutputFormatter, Microsoft.AspNetCore.Mvc.Formatters.StreamOutputFormatter, Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonOutputFormatter
dbug: Microsoft.AspNetCore.Mvc.Infrastructure.DefaultOutputFormatterSelector[6]
Attempting to select an output formatter based on Accept header 'application/json'.
dbug: Microsoft.AspNetCore.Mvc.Infrastructure.DefaultOutputFormatterSelector[2]
Selected output formatter 'Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonOutputFormatter' and content type 'application/json' to write the response.
info: Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor[1]
Executing ObjectResult, writing value of type 'ApiHosteleriaMobile.Entities.Responses.LoginResponse'.
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2]
Executed action ApiHosteleriaMobile.Controllers.LoginController.ValidateLogin (ApiHosteleriaMobile) in 9455.4859ms
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
Executed endpoint 'ApiHosteleriaMobile.Controllers.LoginController.ValidateLogin (ApiHosteleriaMobile)'
dbug: Microsoft.AspNetCore.Server.Kestrel[9]
Connection id "0HMGRQRK5P7RA" completed keep alive response.
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 9577.0655ms 200 application/json; charset=utf-8
dbug: Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets[6]
Connection id "0HMGRQRK5P7RA" received FIN.
dbug: Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets[7]
Connection id "0HMGRQRK5P7RA" sending FIN because: "The client closed the connection."
dbug: Microsoft.AspNetCore.Server.Kestrel[10]
Connection id "0HMGRQRK5P7RA" disconnecting.
dbug: Microsoft.AspNetCore.Server.Kestrel[2]
Connection id "0HMGRQRK5P7RA" stopped.
Strings are not decoded when received as application/json
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1861
Ktor Version and Engine Used (client or server and name)
io.ktor:ktor-client:1.3.2
io.ktor:ktor-client-cio:1.3.2
io.ktor:ktor-client-json:1.3.2
io.ktor:ktor-client-gson:1.3.2
Describe the bug
When requesting a simple string (e.g. foo
) from a ASP.NET Core Web-API the server returns the simple string encoded as Json string ("foo"
) with mime type application/json
.
The ktor client does not use the configured serializer (e.g. GsonSerializer
) to deserialize the response body, but rather directly return the string "foo"
with the quotes to the requesting code. This is a bigger problem when the simple string contains character which must be encoded in json strings like "
or LF. Then the string in the HTTP response look like "\"\n"
and ktor returns the string "\"\n"
(this is a string with length 6) but it should be a string with two characters, the first character a \
and the second a LF.
To Reproduce
Steps to reproduce the behavior:
- Write the following
client = HttpClient {
install(JsonFeature) {
serializer = GsonSerializer()
}
}
val simpleString = client.get<String>(someUrl)
println(simpleString)
- The server at
someUrl
must respond with a HTTP response like following:
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
"foo\"\n"
- See
"foo\"\n"
printed as output
Expected behavior
only foo"
should be printed (two line breaks)
Can I logging single line?
Hi There.
I'am try to using ktor-client-logging-jvm
.
This logging module display multiple line like this and so many line.
HttpClient: REQUEST: http://localhost:8081/testPost
HttpClient: METHOD: HttpMethod(value=POST)
HttpClient: COMMON HEADERS
HttpClient: -> Accept: application/json
HttpClient: -> Accept-Charset: UTF-8
HttpClient: -> User-Agent: Apache-HttpAsyncClient/4.5.3(npay-point)
HttpClient: CONTENT HEADERS
HttpClient: -> Content-Length: 12
HttpClient: -> Content-Type: text/plain; charset=UTF-8
HttpClient: BODY Content-Type: text/plain; charset=UTF-8
HttpClient: BODY START
HttpClient: asdfasdfasdf
HttpClient: BODY END
HttpClient: RESPONSE: 502
HttpClient: METHOD: HttpMethod(value=POST)
HttpClient: FROM: http://localhost:8081/testPost
HttpClient: COMMON HEADERS
HttpClient: -> Connection: keep-alive, keep-alive
HttpClient: -> Content-Length: 11
HttpClient: -> Content-Type: text/plain;charset=UTF-8
HttpClient: -> Date: Fri, 04 Dec 2020 08:14:04 GMT
HttpClient: -> Keep-Alive: timeout=60
HttpClient: BODY Content-Type: text/plain; charset=UTF-8
HttpClient: BODY START
HttpClient: Post Result
HttpClient: BODY END
Can I logging this to single line like this?
Request : Url[http....], Headers, body, params
Response : Header, body
and if logging with elapsed time. it is very helpful.
Thank u.
HttpResponse.bodyAsChannel should not be converted by ContentNegotiation
HttpResponse.bodyAsChannel should not try to convert body to object using jackson.
Run following code, expected: no exception, actual: Caused by: io.ktor.serialization.JsonConvertException: Illegal json parameter found
import io.ktor.client.*
import io.ktor.client.engine.apache.*
import io.ktor.client.plugins.contentnegotiation.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import io.ktor.http.*
import io.ktor.serialization.jackson.*
import io.ktor.server.application.*
import io.ktor.server.engine.*
import io.ktor.server.jetty.*
import io.ktor.server.response.*
import io.ktor.server.routing.*
import kotlinx.coroutines.runBlocking
fun main() {
embeddedServer(Jetty, 8787) {
routing {
get("/") {
call.respondText("{ }", ContentType.Application.Json, HttpStatusCode.OK)
}
}
}.apply { start() }
val client = HttpClient(Apache) {
install(ContentNegotiation) {
jackson {}
}
}
runBlocking {
val response = client.get("http://localhost:8787")
response.bodyAsChannel()
}
}
Ios: NullPointerException when query parameters contain cyrillic symbols in values
I execute a request from the application to ios and pass a string in Cyrillic to the query, I get an error when executing the query. if the string consists of Latin characters, then everything is fine. There is no such problem on the Android platform.
A native application with the Darwin engine doesn't make a request
To reproduce the issue:
- Clone documentation repo: https://github.com/ktorio/ktor-documentation
- Open the
codeSnippets
project in IDEA: https://github.com/ktorio/ktor-documentation/tree/2.0.0-kmm-tutorial/codeSnippets - Run the client-engine-darwin sample:
./gradlew :client-engine-darwin:runReleaseExecutableNative
=> The application is hanging in this state:
66% EXECUTING
> :client-engine-darwin:runReleaseExecutableNative
Version: 2.0.0-eap-327
P.S. The same example for Curl
works fine: https://github.com/ktorio/ktor-documentation/blob/2.0.0-kmm-tutorial/codeSnippets/snippets/client-engine-curl/src/nativeMain/kotlin/Main.kt
Darwin and Kotlin/JS: "List has more than one element" error when header like Content-type is duplicated in a response
When I try to fetch html and analyze it in Multiplatform Mobile share lib, I got a crash in iOS, but work well in Android.
iOS error msg: IllegalArgumentException("List has more than one element.")
URL: https://mp.weixin.qq.com/s/UJipNKgGPzZ1iPJBAaLJXw
so I compile the ktor source code and debug inside, and then I foud out the reason is that the response header "content-type" has a repeated key-value entry,As shown below:
In this case, iOS native network lib will merge them into one key-value entry like {"content-type" : "text/html; charset=UTF-8 , text/html; charset=UTF-8" } .
But on the other hand, ContentType.parse() will parse "content-type" header and get two HeaderValue, after that it invoke Colloection.single() to check the result, then a IllegalArgumentException("List has more than one element.") will be throw.
I checked the implementation of other platforms,browserjs have the same issue.
UninitializedPropertyAccessException in the handleResponseExceptionWithRequest when request or response are accessed through HttpClientCall
To reproduce run the following code:
val client = HttpClient {
expectSuccess = true
HttpResponseValidator {
handleResponseExceptionWithRequest { exception, request ->
request.call.request.url // for request.call.response.status it happens too
}
}
}
client.get("https://httpbin.org/404").bodyAsText()
As a result, an unexpected UninitializedPropertyAccessException
is thrown:
Exception in thread "main" kotlin.UninitializedPropertyAccessException: lateinit property request has not been initialized
at io.ktor.client.call.HttpClientCall.getRequest(HttpClientCall.kt:39)
at ClientKt$main$client$1$1$1.invokeSuspend(client.kt:44)
at ClientKt$main$client$1$1$1.invoke(client.kt)
at ClientKt$main$client$1$1$1.invoke(client.kt)
at io.ktor.client.plugins.HttpCallValidator.processException(HttpCallValidator.kt:53)
at io.ktor.client.plugins.HttpCallValidator.access$processException(HttpCallValidator.kt:39)
at io.ktor.client.plugins.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:128)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:141)
at io.ktor.util.pipeline.SuspendFunctionGun.access$resumeRootWith(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:58)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:138)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:112)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(SuspendFunctionGun.kt:14)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:62)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:178)
at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:166)
at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:397)
at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:431)
at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl$default(CancellableContinuationImpl.kt:420)
at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:328)
at kotlinx.coroutines.ResumeAwaitOnCompletion.invoke(JobSupport.kt:1412)
at kotlinx.coroutines.JobSupport.notifyCompletion(JobSupport.kt:1519)
at kotlinx.coroutines.JobSupport.completeStateFinalization(JobSupport.kt:323)
at kotlinx.coroutines.JobSupport.finalizeFinishingState(JobSupport.kt:240)
at kotlinx.coroutines.JobSupport.tryMakeCompletingSlowPath(JobSupport.kt:906)
at kotlinx.coroutines.JobSupport.tryMakeCompleting(JobSupport.kt:863)
at kotlinx.coroutines.JobSupport.makeCompletingOnce$kotlinx_coroutines_core(JobSupport.kt:828)
at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:100)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
The original exception is swallowed by "No request transformation found" exception when request body is serializable object
I wrote custom plugin takes care about internet connection. If it's no connection it throws an exception. On 1.6.7 it worked as expected.
I expect that exception won't be handled by ktor internal logic. Now it looks like no exception had been thrown at all.
I debug this, onNoConnection block calls as usual.
class ConnectionStatePlugin private constructor(
private val isConnected: () -> Boolean,
private val onNoConnection: () -> Unit
) {
class Config {
var checkConnection: () -> Boolean = { false }
var onNoConnection: () -> Unit = { throw Exception() }
fun build() = ConnectionStatePlugin(
isConnected = checkConnection,
onNoConnection = onNoConnection
)
}
companion object Plugin : HttpClientPlugin<Config, ConnectionStatePlugin> {
override val key = AttributeKey<ConnectionStatePlugin>("ConnectionStatePlugin")
override fun prepare(block: Config.() -> Unit) = Config().apply(block).build()
override fun install(plugin: ConnectionStatePlugin, scope: HttpClient) {
scope.requestPipeline.intercept(HttpRequestPipeline.State) {
with(plugin) {
if (!isConnected()) {
onNoConnection()
}
}
}
}
}
}
install(ConnectionStatePlugin) {
checkConnection = { connectionStateChecker.isOnline() }
onNoConnection = { throw NoInternetException() }
}
Core
Kotlin/Native: testApplication's client sometimes fails to receive ByteArray response from a route
Tested on Kotlin/Native with ktor-server-test-host:2.0.0
dependency.
I'm defining a Route that returns a ByteArray representing a JPEG image to use it as a stub for some external service that would be called by a Ktor Client.
testApplication {
routing {
imageStub(imageBytes)
}
client.get("image.jpeg")
}
private fun Route.imageStub(imageBytes: ByteArray) {
get("image.jpeg") {
call.respondBytes(ContentType.Image.JPEG) { imageBytes }
}
}
The client that is available in testApplication
block sometimes fails to complete the request with IllegalStateException:
kotlin.IllegalStateException: Expected 2305888, actual 2305632
It's reproducible on Linux and Mac. I'm attaching a sample project demonstrating the issue.
Invalid response without error
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1883
Ktor Version and Engine Used (client or server and name)
io.ktor:ktor-server-cio:1.3.2
Describe the bug
Strings like "a", "H" are invalid responses(see https://tools.ietf.org/html/rfc2616#section-6)
To Reproduce
Steps to reproduce the behavior:
Run code:
import io.ktor.http.cio.parseResponse
import io.ktor.utils.io.ByteReadChannel
import kotlinx.coroutines.runBlocking
fun main() {
listOf("A", "H", "a")
.forEach { test(it) }
}
fun test(http: String) {
val response = runBlocking { parseResponse(ByteReadChannel(http)) }!!
println("status=${response.status};statusText={${response.statusText}};headers=[${response.headers}];version=${response.version}")
}
Expected behavior
An error expected - exception or null result.
Invalid HTTP version should fail
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1866
Ktor Version and Engine Used (client or server and name)
io.ktor:ktor-server-cio:1.3.2
Describe the bug
Request should include HTTP version(see https://tools.ietf.org/html/rfc2616#section-3.1). Also leading zeros are prohibited.
To Reproduce
Steps to reproduce the behavior:
Run code:
import io.ktor.http.cio.parseRequest
import io.ktor.utils.io.ByteReadChannel
import kotlinx.coroutines.runBlocking
fun main() {
listOf(
"""
GET / HTTP/1.6
Host: www.example.com
""".trimIndent(),
"""
GET / HTPT/1.1
Host: www.example.com
""".trimIndent(),
"""
GET / _
Host: www.example.com
""".trimIndent(),
"""
GET / HTTP/1.01
Host: www.example.com
""".trimIndent()
).forEach {
test(it)
}
}
fun test(http: String) {
val request = runBlocking { parseRequest(ByteReadChannel(http)) }!!
println("method=${request.method};version=${request.version};uri={${request.uri}};headers=[${request.headers}]")
}
Expected behavior
An error expected - exception or null result.
The colon after the host parameter requires a port
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1864
Ktor Version and Engine Used (client or server and name)
io.ktor:ktor-server-cio:1.3.2
Describe the bug
The colon after the host parameter requires a port(see https://tools.ietf.org/html/rfc3986#section-3.2.3)
To Reproduce
Steps to reproduce the behavior:
Run code:
import io.ktor.http.cio.parseRequest
import io.ktor.utils.io.ByteReadChannel
import kotlinx.coroutines.runBlocking
fun main() {
test(
"""
GET / HTTP/1.1
Host: www.example.com:
""".trimIndent()
)
}
fun test(http: String) {
val request = runBlocking { parseRequest(ByteReadChannel(http)) }!!
println("method=${request.method};uri={${request.uri}};headers=[${request.headers}]")
}
Expected behavior
An error expected - exception or null result.
Docs
How to disable automatic module loading within testApplication?
The documentation (https://ktor.io/docs/testing.html#auto) states that you
can disable loading modules by customizing an environment for tests
However, it does not explain how the environment exactly has to look like to disable automatic loading.
Document how to enable/disable HTTP/2 for different client engines
https://ktor.io/docs/http-client-engines.html#limitations
val javaClient = HttpClient(Java) {
engine {
config {
sslContext(SslSettings.getSslContext())
version(java.net.http.HttpClient.Version.HTTP_2)
}
}
}
val okHttpClient = HttpClient(OkHttp) {
engine {
config {
sslSocketFactory(SslSettings.getSslContext()!!.socketFactory, SslSettings.getTrustManager())
protocols(listOf(Protocol.HTTP_1_1))
}
}
}
Document a new memory model in KMM tutorial
Make client docs less JVM-centric
https://jbs.zendesk.com/agent/tickets/4049802
- https://ktor.io/docs/client-dependencies.html - mention multiplatform and common source set
- add links from topics for specific plugins
Difference between various http client properties
Can someone tell me difference between maxConnectionsCount
, maxConnectionsPerRoute
, pipelineMaxSize
in http client with CIO engine.
Also what is the role of threadsCount
with coroutines in CIO engine ? The kotlin doc inside :
maxConnectionsCount = Maximum allowed connections count.
maxConnectionsPerRoute = Maximum connections per single route.
pipelineMaxSize = Maximum number of requests per single pipeline.
threadsCount = Network threads count advice.
It is crisp, and I understand it, but I also assume here. I would like :
- A better explanation.
- Which of these properties have a relationship with another ?
- How high can these values be set, and what are the consequences of those, for example in regards to resources (like cpu, memory, file descriptors etc) and performance (like can client choke, have a big queue, make app crash, make request durations worse etc) ?
- Is there a recommended threshold/ or tuning ?
Thank you :)
Documentation about how to configure libcurl on Windows
There should be a documentation about how to configure libcurl on Windows using minGW and Cygwin to be able to use ktor-client-curl
engine.
The issue KTOR-3989 if resolved should be documented too.
Also, some information about troubleshooting common problems would be helpful.
IncorrectDereferenceException when trying to create HttpClient from background thread on iOS
After updating to Ktor 2.0.1 from 1.6.8, creating HttpClient on non main scope, e.g. like this:
CoroutineScope(Dispatchers.Default).launch {
val client = HttpClient { }
println("client $client")
}
Causes the following crash:
Uncaught Kotlin exception: kotlin.native.IncorrectDereferenceException: Trying to access top level value not marked as @ThreadLocal or @SharedImmutable from non-main thread
2022-05-02 00:44:08.016506+0700 iosApp[35807:564602] [Unknown process name] copy_read_only: vm_copy failed: status 1.
at 0 iosApp 0x102f17767 kfun:kotlin.Throwable#<init>(kotlin.String?){} + 95
at 1 iosApp 0x102f1122b kfun:kotlin.Exception#<init>(kotlin.String?){} + 91
at 2 iosApp 0x102f113df kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 91
at 3 iosApp 0x102f2318b kfun:kotlin.native.IncorrectDereferenceException#<init>(kotlin.String){} + 91
at 4 iosApp 0x102f4129f ThrowIncorrectDereferenceException + 111
at 5 iosApp 0x1031d4b87 CheckGlobalsAccessible + 43
at 6 iosApp 0x10316635b kfun:io.ktor.client.plugins#<get-PLUGIN_INSTALLED_LIST>(){}io.ktor.util.AttributeKey<io.ktor.util.Attributes> + 31
at 7 iosApp 0x103148423 kfun:io.ktor.client.HttpClientConfig.install$<anonymous>_1-3#internal + 283
at 8 iosApp 0x103148c6b kfun:io.ktor.client.HttpClientConfig.$install$<anonymous>_1-3$FUNCTION_REFERENCE$22.invoke#internal + 95
at 9 iosApp 0x103148d77 kfun:io.ktor.client.HttpClientConfig.$install$<anonymous>_1-3$FUNCTION_REFERENCE$22.$<bridge-UNNN>invoke(-1:0){}#internal + 95
at 10 iosApp 0x1031475a3 kfun:io.ktor.client.HttpClientConfig#install(io.ktor.client.HttpClient){} + 699
at 11 iosApp 0x1031428c7 kfun:io.ktor.client.HttpClient#<init>(io.ktor.client.engine.HttpClientEngine;io.ktor.client.HttpClientConfig<out|io.ktor.client.engine.HttpClientEngineConfig>){} + 3167
at 12 iosApp 0x1031430ab kfun:io.ktor.client.HttpClient#<init>(io.ktor.client.engine.HttpClientEngine;io.ktor.client.HttpClientConfig<out|io.ktor.client.engine.HttpClientEngineConfig>;kotlin.Boolean){} + 135
at 13 iosApp 0x103145bf7 kfun:io.ktor.client#HttpClient(io.ktor.client.engine.HttpClientEngineFactory<0:0>;kotlin.Function1<io.ktor.client.HttpClientConfig<0:0>,kotlin.Unit>){0§<io.ktor.client.engine.HttpClientEngineConfig>}io.ktor.client.HttpClient + 635
at 14 iosApp 0x10318132b kfun:io.ktor.client#HttpClient(kotlin.Function1<io.ktor.client.HttpClientConfig<*>,kotlin.Unit>){}io.ktor.client.HttpClient + 379
at 15 iosApp 0x1031989d7 kfun:com.example.myapplication.Greeting.$<init>$lambda-1COROUTINE$0.invokeSuspend#internal + 275
at 16 iosApp 0x102f1b5eb kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 527
at 17 iosApp 0x103056893 kfun:kotlinx.coroutines.DispatchedTask#run(){} + 2659
at 18 iosApp 0x10301460f kfun:kotlinx.coroutines.EventLoopImplBase#processNextEvent(){}kotlin.Long + 663
at 19 iosApp 0x10306cfbf kfun:kotlinx.coroutines#runEventLoop(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>){} + 695
at 20 iosApp 0x103072f83 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.start$lambda-0#internal + 295
at 21 iosApp 0x1030730a7 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$120.invoke#internal + 67
at 22 iosApp 0x103073197 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$120.$<bridge-UNN>invoke(){}#internal + 67
at 23 iosApp 0x102f25a43 WorkerLaunchpad + 187
at 24 iosApp 0x1031ec95b _ZN6Worker19processQueueElementEb + 1951
at 25 iosApp 0x1031ec147 _ZN12_GLOBAL__N_113workerRoutineEPv + 99
at 26 libsystem_pthread.dylib 0x1cba8a90f _pthread_start + 115
at 27 libsystem_pthread.dylib 0x1cba85b1b thread_start + 7
Sample project attached
Documentation for migration of Authentication server plugin
The AuthenticatePhase
and ChallengePhase
public phases were effectively replaced with corresponding internal hooks so users who relied on those phases are confused about migration.
https://kotlinlang.slack.com/archives/C0A974TJ9/p1651080437152879
https://kotlinlang.slack.com/archives/C0A974TJ9/p1650208125786219
Add sample for the AuthenticationChecked hook
Documentation for appending query parameters for URL in the DefaultRequest
In Ktor 1.6.8 there is the parameter
method for the HttpRequestBuilder
that adds query parameters to a request URL. In the Ktor 2.0.0, we build a request in the scope of the DefaultRequestBuilder
that doesn't have such a method so users are confused.
https://kotlinlang.slack.com/archives/C0A974TJ9/p1644019110608419
https://kotlinlang.slack.com/archives/C0A974TJ9/p1649848172192389
https://stackoverflow.com/questions/72042828/ktor-parameters-with-defaultrequest-2-x
Infrastructure
Unable to build project on Mac
Hi,
I'm unable to build the project on Mac:
*Execution failed for task ':ktor-client:ktor-client-curl:cinteropLibcurlMacosArm64'.
*
I have installed curl & ncurses.
Any idea what might be the issue?
Thanks!
IntelliJ IDEA 2021.2.4 (Ultimate Edition)
Build #IU-212.5712.43, built on December 21, 2021
Runtime version: 11.0.13+8-b1504.49 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 11.6.5
GC: G1 Young Generation, G1 Old Generation
Memory: 4094M
Cores: 16
Non-Bundled Plugins: com.intellij.nativeDebug (212.5284.19), org.jetbrains.kotlin (212-1.6.21-release-334-IJ5457.46), kotest-plugin-intellij (1.1.49-IC-2021.2.3)
Kotlin: 212-1.6.21-release-334-IJ5457.46
IntelliJ IDEA Plugin
Generator performance: make requests from IDE only if user clicks something. Otherwise -- show static (maybe outdated) page
We are getting more users and generator gets higher RPS rate which leads to memory usage growth.
Most of the requests are first-page requests from IDE (as logs show)
We need to consider limiting requests from IDE via hardcoding first page content and only make requests in background in case user really does something.
Samples
The HttpBin sample doesn't compile
To reproduce run the ./gradlew run
command in the HttpBin sample directory.
Server
When returning a String, content negotiation is ignored
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/985
Ktor Version
1.1.3
Ktor Engine Used(client or server and name)
Server - Jetty
JVM Version, Operating System and Relevant Context
Azul 10, OSX / Debian Linux
Feedback
Currently, when you respond with a string, even if you have a header of Accept=application/json
, due to fun ApplicationSendPipeline.installDefaultTransformations()
, TextContent
is always returned. This means you can't return a bare json string, ie "foo"
.
The special case of String
is handled here:
@KtorExperimentalAPI
fun PipelineContext<Any, ApplicationCall>.transformDefaultContent(value: Any): OutgoingContent? = when (value) {
is OutgoingContent -> value
is String -> {
val contentType = call.defaultTextContentType(null)
TextContent(value, contentType, null)
}
is ByteArray -> {
ByteArrayContent(value)
}
is HttpStatusCode -> {
HttpStatusCodeContent(value)
}
is URIFileContent -> {
if (value.uri.scheme == "file")
LocalFileContent(File(value.uri))
else
null
}
else -> null
}
"Application started" is never printed
We have the following block in the io.ktor.server.engine.BaseApplicationEngine
environment.monitor.subscribe(ApplicationStarted) { val finishedAt = currentTimeMillisBridge() val elapsedTimeInSeconds = (finishedAt - info.initializedStartAt) / 1_000.0 if (info.isFirstLoading) { environment.log.info("Application started in $elapsedTimeInSeconds seconds.") info.isFirstLoading = false } else { environment.log.info("Application auto-reloaded in $elapsedTimeInSeconds seconds.") } }
The first section of the if statement is never actually called because the isFirstLoading
is always false
Route's path parameters are empty when ApplicationCall.authentication is first accessed in a different ApplicationCall context
https://github.com/rasharab/KtorBugWithMonitoring
The above is a clear reproduction of this bug (with appropriate test cases to illustrate my point).
This is a regression we observed as of Ktor 2.0.0 (was working with the betas).
The gist of the bug is that if we add an MDC context for the application JWTPrincipal, that somehow causes the application call parameters to be reset during Authentication.
This bug is only present if we attempt to do the below block in Monitoring.kt
In Monitoring.kt
we have:
mdc("identityId") {
it.principal<JWTPrincipal>()?.subject ?: "No subject"
}
In Security.kt
authentication {
jwt("bearerAuth") {
verifier(
...
)
validate { credential ->
val teamId = this.parameters["teamId"]
if (teamId == null) {
println("TeamID is unexpectedly null!")
throw Exception("TeamID is null when it should't be")
}
if (credential.payload.audience.contains("audience")) JWTPrincipal(credential.payload) else null
}
}
}
The teamId parameter should always be populated based off the below route, but it isn't when monitoring the JWTPrincipal is enabled.
Our routing is very simple:
fun Application.configureRouting() {
routing {
authenticate("bearerAuth") {
get("/teams/{teamId}") {
call.respondText("Hello World!")
}
}
}
}
Resources: builder methods return routes with PathSegmentConstantRouteSelector instead of HttpMethodRouteSelector
The Locations plugin's functions which add routes with methods (e.g. get<MyDataClass> { ... }
do not return the correct route.
Expected: they should return the route whose selector is the GET method selector (similar to the regular non-Locations get
).
Actual: They return the parent of the expected route, meaning that we get a route that matches only the path and not the method.
This is a problem for libraries that rely on the returned route to do other things (in the case of a library I author, Koa, I need the returned route to provide all the information of a path to turn it, via a DSL, into an OpenAPI Operation). The problem comes from these lines (and their equivalents with other HTTP methods): https://github.com/ktorio/ktor/blob/809ebf183c12fb580fdbfa33bf6c13422f4fe19b/ktor-server/ktor-server-plugins/ktor-server-locations/jvm/src/io/ktor/server/locations/Location.kt#L82-L91 which should return the result of method(HttpMethod.Get)
instead of returning the return value of location
.
Shared
JacksonWebsocketContentConverter.deserialize just doesn't work
In io.ktor:ktor-serialization-jackson:2.0.0
, method JacksonWebsocketContentConverter.deserialize
is not working at all. This prevent receiving anything with receiveDeserialized
in a websocket session.
Turns out, after a quick inspection the problem comes from line 37 where the frame is decoded into a string.
return withContext(Dispatchers.IO) {
val data = charset.newDecoder().decode(buildPacket { content.readBytes() })
objectmapper.readValue(data, objectmapper.constructType(typeInfo.reifiedType))
}
Here, buildPacket
expects a lambda in the context of BytePacketBuilder
but no builder method is called. content.readBytes()
throws the ByteArray
into the wind and the ByteReadPacket
is always ByteReadPacket.Empty
. The ObjectMapper will then always fail :
2022-04-28 17:10:55.132 [eventLoopGroupProxy-3-3] ERROR ktor.application - Websocket handler failed
io.ktor.serialization.JsonConvertException: Illegal json parameter found
at io.ktor.serialization.jackson.JacksonWebsocketContentConverter.deserialize(JacksonWebsocketContentConverter.kt:41)
at io.ktor.serialization.jackson.JacksonWebsocketContentConverter$deserialize$1.invokeSuspend(JacksonWebsocketContentConverter.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.ktor.server.netty.EventLoopGroupProxy$Companion.create$lambda-1$lambda-0(NettyApplicationEngine.kt:263)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
at [Source: (String)""; line: 1, column: 0]
at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4765)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4667)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3629)
at io.ktor.serialization.jackson.JacksonWebsocketContentConverter$deserialize$2.invokeSuspend(JacksonWebsocketContentConverter.kt:38)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:39)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
This can be fixed by calling writeFully(content.readBytes())
or by rewriting JacksonWebsocketContentConverter.deserialize
in a similar fashion to KotlinxWebsocketSerializationConverter
.
Test Infrastructure
Test fails when using externalServices block
This test will fail when /src/main/resource∕application.conf
has an unnsatisfied substitution {SOME_ENV}
even if the test is set to use a file src/test/resources/application-test.conf
. Adding the externalServices
block is was causing the behavior of looking for /src/main/resource∕application.conf
in the test.
I would expect the file given for the test is the one that should be requiered
@Test
fun testAuth() = testApplication {
environment {
config = ApplicationConfig("application-test.conf")
}
externalServices {
//For some reason this one requiers the original config from resources
hosts("http://example.com"){
}
}
Other
Update Jackson to 2.13.3
Revert Dokka to 1.6.10 due to Publication Freeze
Update Netty to 4.1.77.Final
Default request without explicit port sets port 80 for all requests
after upgrading from 2.0.0 to 2.0.1 basically every request causes an exception, what i'm seeing is that requests now append the port in every url:
2.0.0:
https://my-website.com/v2/api
2.0.1:
https://my-website.com:80/v2/api
API Docs reference RFCs. Better to reference our own documentation
Many plugins reference RFCs, such as the one for ForwardedHeaders or AuthScheme, and yet it would make sense to reference our own documentation (which in turn actually references RFCs'). This should be applied consistently across the entire code base.