Changelog 2.3 version
2.3.13
released 21st November 2024
Client
CIO: Requests face connection timeouts when executed on the Android main dispatcher
In some cases the engine stops being able to execute any requests, instead failing all of them with a timeout.
I traced this to the semaphores in Endpoint and ConnectionFactory not being released when a request is made from the main thread. Endpoint.releaseConnection() attempts to get the address to be released using InetSocketAddress(host, port), which throws android.os.NetworkOnMainThreadException. This gets caught and doesn't cause a crash, but it does skip the semaphore releases, and so over time all of their permits get used up and once they run out, all subsequent requests will get stuck on these semaphores, never actually getting a permit and eventually failing with a timeout.
To reproduce, create a client using the cio engine, and set the engine's maxConnectionsCount and endpoint.maxConnectionsPerRoute to some smaller number, so you don't have to wait forever. Then make a bunch of requests from the main thread and wait until they start failing.
Now whether or not making requests from the main thread has any merit is perhaps a question to be considered, but this is not acceptable behavior either way - ktor should either throw an exception before even attempting the request, or do it properly. I personally don't see a reason to restrict this, given that the main thread is not actually used for the io (not intentionally anyway).
I include a small patch which fixes the issue for me here for your consideration. I can't currently test this very well with the latest main branch, but this has worked fine in production with ktor 2.3.8 for a few weeks now. The idea is simple: Just store the resolved address in a variable, so we can use the exact same instance later on instead of resolving it again.
diff --git a/ktor-client/ktor-client-cio/jvmAndNix/src/io/ktor/client/engine/cio/Endpoint.kt b/ktor-client/ktor-client-cio/jvmAndNix/src/io/ktor/client/engine/cio/Endpoint.kt
index cd099505e..81c1db0e5 100644
--- a/ktor-client/ktor-client-cio/jvmAndNix/src/io/ktor/client/engine/cio/Endpoint.kt
+++ b/ktor-client/ktor-client-cio/jvmAndNix/src/io/ktor/client/engine/cio/Endpoint.kt
@@ -37,6 +37,8 @@ internal class Endpoint(
private val deliveryPoint: Channel<RequestTask> = Channel()
private val maxEndpointIdleTime: Long = 2 * config.endpoint.connectTimeout
+ private lateinit var address: InetSocketAddress
+
private val timeout = launch(coroutineContext + CoroutineName("Endpoint timeout($host:$port)")) {
try {
while (true) {
@@ -59,6 +61,7 @@ internal class Endpoint(
callContext: CoroutineContext
): HttpResponseData {
lastActivity.value = getTimeMillis()
+ address = InetSocketAddress(host, port)
if (!config.pipelining || request.requiresDedicatedConnection()) {
return makeDedicatedRequest(request, callContext)
@@ -201,8 +204,6 @@ internal class Endpoint(
try {
repeat(connectAttempts) {
- val address = InetSocketAddress(host, port)
-
val connect: suspend CoroutineScope.() -> Socket = {
connectionFactory.connect(address) {
this.socketTimeout = socketTimeout
@@ -284,7 +285,6 @@ internal class Endpoint(
}
private fun releaseConnection() {
- val address = InetSocketAddress(host, port)
connectionFactory.release(address)
connections.decrementAndGet()
}
Core
"java.lang.IllegalArgumentException: Failed requirement." in SelectorManagerSupport
Some of my Android users are getting the following exception:
Fatal Exception: java.lang.IllegalArgumentException: Failed requirement.
at io.ktor.network.selector.SelectorManagerSupport.select(SelectorManagerSupport.java:34)
at io.ktor.network.sockets.DatagramSocketImpl.receiveSuspend(DatagramSocketImpl.java:77)
at io.ktor.network.sockets.DatagramSocketImpl.receiveImpl(DatagramSocketImpl.java:66)
at io.ktor.network.sockets.DatagramSocketImpl.access$receiveImpl(DatagramSocketImpl.java:16)
at io.ktor.network.sockets.DatagramSocketImpl$receiver$1.invokeSuspend(DatagramSocketImpl.java:40)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(BaseContinuationImpl.java:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.java:106)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.java:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.java:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.java:665)
There are just a few users who get this exception. I am not able to reproduce the issue myself so it would be a bit hard to create a sample project. I am using Ktor UDP sockets which seem to be the cause by looking at the stack trace.
My app uses a manually built version of Ktor based on the following commit: https://github.com/Thomas-Vos/ktor/tree/94fab7c9411414506c27ba3e120af64210535d5e
Any ideas why this could be happening? It would seem to me that this type of exception should never happen.
Server
Replace custom withTimeout implementation using WeakTimeoutQueue with coroutines.withTimeout
Check, if this custom WeakTimeoutQueue is still needed with coroutines 1.6.0 or could be replaced by the existing coroutines.withTimeout function.
Reason:
- remove "outdated" workarounds and use more coroutines apis
- coroutines.withTimeout does not need a GMTClock, but uses CoroutineScheduler
Other
Add watchosDeviceArm64 target
According to the doc, target `watchosDeviceArm64` is missing
{width=70%}
And the project with this target enabled can't be built
{width=70%}
See https://youtrack.jetbrains.com/issue/KT-53107/Add-arm64-support-for-watchOS-targets-Xcode-14 for reference
io.ktor.util.TextKt.chomp doesn't work on strings with more than one character
chomp assumes the input string is exactly one character, which is understandable given its usage within Ktor, but since it's public, I would expect it to work with any non-empty separator.
The fix for this is trivial, and I would make a PR for it myself, but Gradle gets consumed by OOM before Ktor build finishes on my machine. :/
2.3.12
released 21st June 2024
Client
OpenTelemetry error spans for successful requests
The new version of OpenTelemetry produces error spans for successful responses with exception. I suppose that it happens for all http requests
kotlinx.coroutines.JobCancellationException: JobImpl has completed normally; job=JobImpl{Completed}@2da13f92
at kotlinx.coroutines.JobSupport.getCancellationException(JobSupport.kt:422)
at io.opentelemetry.instrumentation.ktor.v2_0.client.KtorClientTracing$Companion$installSpanEnd$1$1.invokeSuspend(KtorClientTracing.kt:101)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith$$$capture(ContinuationImpl.kt:33)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
OpenTelemetry: Incorrect end time of span when receiving response body
Currently, OpenTelemetry client tracing is based on HttpReceivePipeline.After. This phase is called after the receiving of response headers. As a result, we measure incorrect time of request and this difference can be extremely big in the case of log-running calls, for example, SSE.
Core
Embedded Linux device without iso-8859-1 and UTF-16 cannot use ktor-network
I'm trying to use ktor-network to speak UTF-8 to a unix domain socket. The Charsets object on linux native eagerly initializes three charsets using iconv: UTF-8, ISO-8859-1, and UTF-16 (either BE or LE). My embedded Linux device only supports UTF-8, and both ISO-8859-1 and UTF-16BE fail preventing any use of ktor (tested manually using iconv_open).
Since ISO 8859-1 is only used by HTTP headers and I have no idea what UTF-16 is used for, could those be made lazy? Or perhaps all of them should be lazily initialized? Or maybe only UTF-8 should be provided at this layer and the HTTP module should cache its own ISO 8859-1 instance and whatever uses UTF-16 could cache its own instance?
Caused by: kotlin.IllegalArgumentException: Failed to open iconv for charset ISO-8859-1 with error code 22
at 0 example.kexe 0xea833b kfun:kotlin.Throwable#<init>(kotlin.String?){} + 91
at 1 example.kexe 0xea2ec3 kfun:kotlin.Exception#<init>(kotlin.String?){} + 83
at 2 example.kexe 0xea3093 kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 83
at 3 example.kexe 0xea3263 kfun:kotlin.IllegalArgumentException#<init>(kotlin.String?){} + 83
at 4 example.kexe 0x14df68f kfun:io.ktor.utils.io.charsets#checkErrors(kotlinx.cinterop.CPointer<out|kotlinx.cinterop.CPointed>?;kotlin.String){} + 639
at 5 example.kexe 0x14df0c3 kfun:io.ktor.utils.io.charsets.CharsetIconv.<init>#internal + 515
at 6 example.kexe 0x14ded53 kfun:io.ktor.utils.io.charsets.Charsets#<init>(){} + 275
at 7 example.kexe 0x14dec03 kfun:io.ktor.utils.io.charsets.Charsets.$init_global#internal + 147
at 8 example.kexe 0x15dc723 CallInitGlobalPossiblyLock + 487
at 9 example.kexe 0x14dee87 kfun:io.ktor.utils.io.charsets.Charsets#<get-$instance>#static(){}io.ktor.utils.io.charsets.Charsets + 71
Note: I'm actually using 3.0.0-beta-1 but I'm not allowed to change the affected versions for some reason.
Docs
Docs: Update versions in codeSnippets
Most importantly, the kotlin coroutines and serialization versions are wrong and need to be set to the latest available versions.
Gradle needs to be updated to v8.
Methods in FakeTaskRepository in first iteration shouldn't have the suspend keyword
In this tutorial: https://ktor.io/docs/server-integrate-database.html#add-starter-code
The methods ofFakeTaskRepository in the first iteration should not have the suspend keyword.
The code should be as follows:
class FakeTaskRepository : TaskRepository {
private val tasks = mutableListOf(
Task("cleaning", "Clean the house", Priority.Low),
Task("gardening", "Mow the lawn", Priority.Medium),
Task("shopping", "Buy the groceries", Priority.High),
Task("painting", "Paint the fence", Priority.Medium)
)
override fun allTasks(): List<Task> = tasks
override fun tasksByPriority(priority: Priority) = tasks.filter {
it.priority == priority
}
override fun taskByName(name: String) = tasks.find {
it.name.equals(name, ignoreCase = true)
}
override fun addTask(task: Task) {
if (taskByName(task.name) != null) {
throw IllegalStateException("Cannot duplicate task names!")
}
tasks.add(task)
}
override fun removeTask(name: String): Boolean {
return tasks.removeIf { it.name == name }
}
}
Documentation Restructuring: Add new content
Implement the changes necessary as described in Documentation Restructuring from June 2023.
Existing tutorials are to be replaced with the following:
- Getting Started Guide
- Routing and Requests
- Content Negotiation and REST
- Building Web Applications
- Working with WebSockets
- Database with Exposed
- Security with OAuth
- Cloud Deployment and Configuration
Add a new tutorial 'Full Stack development with Kotlin Multiplatform'
The content is available here
Add a new tutorial 'First steps with Kotlin RPC'
Transfer the content from the tutorial 'Fist steps with Kotlin RPC` into the Ktor Docs.
To be placed under a new topic section called 'Integrations'.
Improve the docs by showcasing broadcasting messages to WebSocket clients via SharedFlow
Hello Ktor team,
I've been working with WebSockets in Ktor and noticed an opportunity for improvement in how broadcast messages are handled. Currently, maintaining a list of WebSocket connections for broadcasting can be cumbersome. I'd like to propose using SharedFlow as an alternative approach.
https://ktor.io/docs/server-websockets.html#handle-multiple-session
Current approach:
- Maintain a list of WebSocket connections
- Iterate through the list to send broadcast messages
Proposed improvement:
- Use a
SharedFlowto manage broadcast messages - Each WebSocket connection collects from the
SharedFlow
Benefits:
- Simplified code structure
- Better handling of concurrency
- Improved scalability for multiple connections
- Easier to manage connection lifecycles
I've implemented a proof of concept using SharedFlow that works well. Here's a simplified version of the code:
fun Route.routingWS() {
val messageFlow = MutableSharedFlow<Message>()
val sharedFlow = messageFlow.asSharedFlow()
webSocket("/ws") {
send("You are connected to WebSocket!")
val job = launch {
sharedFlow.collect { message ->
send("Broadcast: ${message.message}")
}
}
try {
incoming.consumeEach { frame ->
if (frame is Frame.Text) {
val receivedText = frame.readText()
val message = Message(receivedText)
messageFlow.emit(message)
send("You said: $receivedText")
}
}
} finally {
job.cancel()
}
}
}
I believe this approach could be beneficial for Ktor users working with WebSockets.
Question : Would you consider incorporating this pattern into the Ktor documentation or examples?
Example project: https://github.com/mbakgun/dcbln24-mbakgun/
I appreciate your consideration.
Samples
Update Gradle version in Ktor-samples
Currently, you get an error if you are trying to import samples projects
{width=70%}
Updating the Gradle version will fix the problem, but I think we can solve this problem in general because all users who want to update the Ktor Gradle plugin will encounter it
@leonid.stashevsky
Server
NoSuchMethodError when using coroutines 1.9.0-RC
To reproduce, run the following code while using the kotlinx-coroutines-core:1.9.0-RC:
embeddedServer(Netty, port = 3333) {}.start()
As a result, the following exception is thrown:
Exception in thread "main" java.lang.NoSuchMethodError: 'void kotlinx.coroutines.internal.LockFreeLinkedListHead.addLast(kotlinx.coroutines.internal.LockFreeLinkedListNode)'
at io.ktor.events.Events.subscribe(Events.kt:24)
at io.ktor.server.engine.BaseApplicationEngine.<init>(BaseApplicationEngine.kt:50)
at io.ktor.server.engine.BaseApplicationEngine.<init>(BaseApplicationEngine.kt:31)
at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:33)
at io.ktor.server.netty.Netty.create(Embedded.kt:18)
at io.ktor.server.netty.Netty.create(Embedded.kt:13)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:111)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:100)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:65)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:40)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer$default(EmbeddedServer.kt:32)
Server: Content-Type header for static js, css and svg resources misses charset
For example:
Static .js files are served with Content-Type: text/javascript instead of Content-Type: text/javascript; charset=utf-8.
Same issue with .css and .svg.
Update dependency on swagger
Now the ktor swagger plugin does not support Open Api 3.1.*. Swagger supports 3.1.
We could update the dependency on swagger in the plugin io.ktor:ktor-server-swagger
2.3.11
released 9th May 2024
Docs
Sessions: documentation snippet lacks import statement
UPD: actually, I wrote initialization code in the wrong lambda. ktor IDEA plugin generates this main function:
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0", module = Application::module)
.start(wait = true)
}
while the documentation snippet here defines configuration inside the lambda parameter:
fun main() {
embeddedServer(Netty, port = 8080) {
install(Sessions)
// ...
}.start(wait = true)
}
But the embeddedServer function has another lambda parameter and I tried to call install(Sessions) there, which resulted in error:
fun main() {
db.apply {}
embeddedServer(Netty, port = 8080, host = "0.0.0.0", module = Application::module) {
// compilation error here:
install(Sessions)
}
.start(wait = true)
}
Looks like the documentation should be aligned with the template or vice versa.
Ktor client with logback-classic on android produces compilation error on build
Hi,
We've been using Ktor client in our android project with a very old version of logback which broke after it was updated to 1.4.6 + Android Gradle Plugin 8.0.0.
The error was the same described in this SO post (2 files found with path 'META-INF/INDEX.LIST' from inputs):
https://stackoverflow.com/questions/75704844/how-to-set-up-ktor-logging-in-android
* What went wrong:
Execution failed for task ':app:mergeDebugJavaResource'.
> A failure occurred while executing com.android.build.gradle.internal.tasks.MergeJavaResWorkAction
> 2 files found with path 'META-INF/INDEX.LIST' from inputs:
- D:\Software\.gradle\caches\modules-2\files-2.1\ch.qos.logback\logback-classic\1.4.5\28e7dc0b208d6c3f15beefd73976e064b4ecfa9b\logback-classic-1.4.5.jar
- D:\Software\.gradle\caches\modules-2\files-2.1\ch.qos.logback\logback-core\1.4.5\e9bb2ea70f84401314da4300343b0a246c8954da\logback-core-1.4.5.jar
Adding a packagingOptions block may help, please refer to
https://developer.android.com/reference/tools/gradle-api/7.4/com/android/build/api/dsl/ResourcesPackagingOptions
for more information
In this SO post Aleksei suggested to use org.slf4j:slf4j-android:1.7.36 SLF4J Android Binding library instead of the logback-classic which worked for us and the guy in the SO post too.
If this is the official recommendation for android could you please update the relevant docs with this library so others can use it straight away.
Thank you!
Update the procedure and screenshots in "Creating a client application" tutorial
https://ktor.io/docs/client-create-new-application.html
IDEA doesn't have the Kotlin Multiplatform template anymore. The project should be created from a Kotlin template instead.
Broken link in the "Integrate a database" topic
On the "Integrate a database with Kotlin, Ktor, and Exposed" page the link tutorial-server-database-integration to the sample leads to the 404 page.
Replace the tutorials under 'Creating a website' with new content
The material for the two pages within 'Creating a Website' has been combined into Building Web Applications. If you want to preserve existing titles and URL's you could still call this Creating An Interactive Website. NB no one would use a framework like Ktor to create a static website as described here. It's the wrong tool for the job,
Replace the 'Creating a WebSocket chat' tutorial with new content
The Creating a Website Chat page is replaced by Working With WebSockets. This new tutorial continues and extends the Task Manager case study from the previous tutorials. So it should be much easier for readers understand.
Add a new tutorial 'Database integration with Exposed'
The content is available here
Replace Creating HTTP API's tutorial with new content
The material from the Creating HTTP API's page is to be split into Understanding Routing and Requests and Content Negotiation and REST.
You could still use the old name and URL for the first of these tutorials. It's close enough.
Test Infrastructure
Test client ignores socket timeout
I would expect the following test to pass, but it looks like socket timeout is ignored by the test client.
class ApplicationTest {
@Test
fun `test socketTimeout`() = testApplication {
routing {
get("/") {
call.respondOutputStream {
write("Hello World".toByteArray())
delay(10000)
}
}
}
val clientWithTimeout = client.config {
install(HttpTimeout) {
socketTimeoutMillis = 1_000
}
}
assertFails {
println(clientWithTimeout.get("/").readBytes().decodeToString())
}
}
}
2.3.10
released 8th April 2024
Client
NodeJS WebSocket client sometimes drops a frame received immediately after handshake
When the server immediately sends a frame back to the client upon handshake, the client sometimes misses it.
Possible root cause
I believe this is due to the fact that the onMessage web socket listener is not registered immediately after the creation of the web socket instance (which performs the handshake). In JS, since things are single-threaded, it's OK to do the following:
val ws = WebSocket(url)
ws.onmessage = { ... }
But in the case of Ktor implementation, there is an awaitConnection() between the handshake and the message listener registration:
https://github.com/ktorio/ktor/blob/2.3.9/ktor-client/ktor-client-core/js/src/io/ktor/client/engine/js/JsClientEngine.kt#L91-L100
val socket: WebSocket = createWebSocket(urlString, request.headers) // performs the handshake
try {
socket.awaitConnection()
} catch (cause: Throwable) {
callContext.cancel(CancellationException("Failed to connect to $urlString", cause))
throw cause
}
val session = JsWebSocketSession(callContext, socket) // registers socket.onmessage in init{} block
I think the presence of this await here allows a frame to arrive before the message event listener is registered, thus dropping the frame.
Reproducer
First, we need to setup a test server that sends a frame upon handshake. For instance, this simple test server uses org.java-websocket:Java-WebSocket:1.5.6:
import org.java_websocket.*
import org.java_websocket.handshake.*
import org.java_websocket.server.*
import java.net.*
import java.nio.*
fun main() {
val port = 12345
val server = EchoWebSocketServer(port)
println("Starting test WS server...")
server.start()
println("Server listening at ws://localhost:$port")
}
internal class EchoWebSocketServer(port: Int = 0) : WebSocketServer(InetSocketAddress(port)) {
override fun onOpen(conn: WebSocket, handshake: ClientHandshake) {
println("Received connection from ${conn.remoteSocketAddress}${conn.resourceDescriptor}")
conn.send("Hello client!")
println("Said hi to the client")
}
override fun onStart() {}
override fun onMessage(conn: WebSocket, message: String?) {}
override fun onMessage(conn: WebSocket, message: ByteBuffer) {}
override fun onError(conn: WebSocket?, ex: Exception?) {}
override fun onClose(conn: WebSocket, code: Int, reason: String?, remote: Boolean) {}
}
Then we can run this test that demonstrates the problem:
import io.ktor.client.*
import io.ktor.client.plugins.websocket.*
import io.ktor.websocket.*
import kotlinx.coroutines.*
import kotlinx.coroutines.test.*
import kotlin.test.*
import kotlin.time.Duration.Companion.minutes
import kotlin.time.Duration.Companion.seconds
private const val port = 12345 // TODO replace with the acutal port used by the test server
private const val nAttempts = 10 // usually the problem occurs after 2-3 iterations
private val frameTimeout = 2.seconds // we can increase this to confirm it's not a matter of time
class SomeJsTest {
@Test
fun testHandshakeResponse() = runTest(timeout = 1.minutes) {
withContext(Dispatchers.Default) { // to avoid delay skipping from runTest (we're testing a real system)
var nMissed = 0
repeat(nAttempts) { attempt ->
val httpClient = HttpClient { install(WebSockets) }
val session = httpClient.webSocketSession("ws://localhost:$port")
try {
println("Expecting immediate hello from the server...")
val helloFrame = withTimeoutOrNull(frameTimeout) {
session.incoming.receive()
}
if (helloFrame == null) {
println("$attempt: MISSED! (Hello frame not received $frameTimeout after handshake)")
nMissed++
} else {
assertIs<Frame.Text>(helloFrame, "should receive a Text frame as echo")
assertEquals("Hello client!", helloFrame.readText(), "should receive echo")
println("$attempt: Got our hello frame this time!")
}
} finally {
session.close()
}
}
if (nMissed > 0) fail("$nMissed/$nAttempts frames were missed")
}
}
}
The output should look like this:
Expecting immediate hello from the server...
0: Got our hello frame this time!
Expecting immediate hello from the server...
1: MISSED! (Hello frame not received 2s after handshake)
Expecting immediate hello from the server...
2: MISSED! (Hello frame not received 2s after handshake)
Expecting immediate hello from the server...
3: Got our hello frame this time!
Expecting immediate hello from the server...
4: Got our hello frame this time!
Expecting immediate hello from the server...
5: Got our hello frame this time!
Expecting immediate hello from the server...
6: Got our hello frame this time!
Expecting immediate hello from the server...
7: MISSED! (Hello frame not received 2s after handshake)
Expecting immediate hello from the server...
8: MISSED! (Hello frame not received 2s after handshake)
Expecting immediate hello from the server...
9: MISSED! (Hello frame not received 2s after handshake)
AssertionError: 5/10 frames were missed
JS browser: "Maximum call stack size exceeded" on HTTP request when targeting es2015
Setting target to es2015 causes JS client to fail in Chrome
Reproducer:
- Create a minimal project with the code below
- Run tests, observe that the tests succeed
- Run
jsBrowserDevelopmentWebpack - Create this html file
<body style="background: black; color: white">
<p>hello world</p>
<script src="/path/to/ktor-bug-project/build/kotlin-webpack/js/developmentExecutable/ktor-bug-project.js"></script>
</body>
- Open this HTML file in Google Chrome and observe the error
build.gradle.kts
plugins {
kotlin("multiplatform") version "2.0.0-Beta5"
}
repositories.mavenCentral()
kotlin {
js(IR) {
@Suppress("OPT_IN_USAGE")
compilerOptions.target.set("es2015")
browser{
binaries.executable()
}
}
sourceSets.jsMain.dependencies {
implementation("io.ktor:ktor-client-js:2.3.9")
}
}
dependencies {
commonTestImplementation(kotlin("test"))
}
main.kt
import io.ktor.client.HttpClient
import io.ktor.client.request.get
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Job
import kotlinx.coroutines.launch
fun main() {
CoroutineScope(Job()).launch {
HttpClient().get("https://www.example.com")
}
}
test.kt
import kotlin.test.Test
class Tests {
@Test
fun runMain() {
main()
}
}
Android: no logs are present in Logcat with `Logger.ANDROID`
Perhaps it's just an issue of documentation, but when I enable logging with HttpClient(Android) no logs is present unless I provide custom logger implementation.
In more details:
When I create HttpClient with Android Engine using either Logger.DEFAULT or Logger.ANDROID no logs is present in the Logcat .
HttpClient(Android) {
if (SdkConfig.isLoggingEnabled) {
install(Logging) {
logger = Logger.DEFAULT
level = LogLevel.ALL
}
}
}
I am adding logging dependency as io.ktor:ktor-client-logging-jvm:1.4.1. As far as I see there is no specific "android" artefact available.
As far as I see both Logger.DEFAULT or Logger.ANDROID uses slf4j for logging, which does nothing on Android by default.
Ideally when Logger.DEFAULT logger is used in Android it should log to Android system log.
And obviously Logger.ANDROID is expected to do this without any further configuration.
P.S. I have not tried yet logging on JVM, but as for me, it make sense to add the mentioning about required slf4j implementation (e.g. logback) in the dependencies there as well.
CIO: "getSubjectAlternativeNames(...) must not be null" when IP-addresses are verified and no SAN in the certificate
My host have only ip and self-signed certificate. My version ktor 2.3.8
My client setup:
client = HttpClient(CIO) {
install(WebSockets){
pingInterval = 5_000
}
engine {
https {
trustManager = object : X509TrustManager {
override fun checkClientTrusted(chain: Array<out X509Certificate>?, authType: String?) { }
override fun checkServerTrusted(chain: Array<out X509Certificate>?, authType: String?) { }
override fun getAcceptedIssuers(): Array<X509Certificate> {
return emptyArray()
}
}
}
}
install(Logging) {
logger = object : Logger {
override fun log(message: String) {
Timber.tag("Socket Log").e(message)
}
}
level = LogLevel.ALL
}
}
But after try connect i have error: certificate.subjectAlternativeNames must not be null
java.lang.NullPointerException: certificate.subjectAlternativeNames must not be null
at io.ktor.network.tls.HostnameUtilsKt.verifyIpInCertificate(HostnameUtils.kt:30)
at io.ktor.network.tls.HostnameUtilsKt.verifyHostnameInCertificate(HostnameUtils.kt:15)
at io.ktor.network.tls.TLSClientHandshake.handleCertificatesAndKeys(TLSClientHandshake.kt:245)
at io.ktor.network.tls.TLSClientHandshake.negotiate(TLSClientHandshake.kt:166)
at io.ktor.network.tls.TLSClientHandshake$negotiate$1.invokeSuspend(Unknown Source:18)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@abda18f, Dispatchers.IO]
Docs
Creating HTTP APIs: Incorrect snippet for serialization plugin
In https://ktor.io/docs/server-create-http-apis.html#dependencies:
In order to use
ktor-serialization-kotlinx-json, we also have to apply theplugin.serializationplugin.
However the snippet below shows how to apply the ktor plugin and not the serialization plugin. The snippet is pointing to {src="snippets/tutorial-http-api/build.gradle.kts" include-lines="5,9-10"}. The serialization plugin is also in that file, but at line 8, and it's for an older version of kotlin.
Don't mind contributing a PR for this but not sure if it should point to that file. Pointers welcome (no pun intended).
Timeout topic: Some links lead to 404 pages
On the https://ktor.io/docs/client-timeout.html#configure_plugin page links to requestTimeoutMillis, connectTimeoutMillis and socketTimeoutMillis lead to the 404 page.
Docs: Update Ktor logo and replace favicon
Currently, the JetBrains logo is shown as a favicon on the browser.
We can use the custom-favicons element with a Ktor logo.
Additionally the Ktor logo needs to be updated.
Gradle Plugin
ShadowJar: Support enabling ZIP64 format to overcome limitation of 65535 entries
Please, help me.
When I use the ktor-build-plugin to execute the shadowJar task, the compiler gives me the following error.
> Task :service:service-api:shadowJar FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':service:service-api:shadowJar'.
> shadow.org.apache.tools.zip.Zip64RequiredException: archive contains more than 65535 entries.
To build this archive, please enable the zip64 extension.
See: https://docs.gradle.org/8.5/dsl/org.gradle.api.tasks.bundling.Zip.html#org.gradle.api.tasks.bundling.Zip:zip64
Server
Inconsistent behavior of Netty and rest engines by returning null or empty string for query parameters without values
Use this code to compare between Netty and CIO and visit http://localhost:8080/?auto, http://localhost:8081/?auto`:
fun main() {
embeddedServer(Netty, port = 8080) {
routing {
get("/") {
call.respondText(this.context.request.queryParameters["auto"].toString())
}
}
}.start()
embeddedServer(CIO, port = 8081) {
routing {
get("/") {
call.respondText(this.context.request.queryParameters["auto"].toString())
}
}
}.start(true)
}
CIO sets the query parameter to null (so it's equivalent to auto not being present at all). Netty sets it to a blank string (so you can detect its presence):
cio-qpm.zip
CallLogging, StatusPages: response logged twice when status handler is used
To reproduce, run the following test:
@Test
fun test() = testApplication {
application {
install(CallLogging)
install(StatusPages) {
status(HttpStatusCode.BadRequest) { call, _ ->
call.respondText { "Bad request" }
}
}
routing {
get {
call.respond(HttpStatusCode.BadRequest)
}
}
}
client.get("/")
}
As a result, the response for the / request is unexpectedly logged twice:
10:01:25.843 [DefaultDispatcher-worker-1 @request#2] INFO io.ktor.test -- 200 OK: GET - / in 137ms
10:01:25.843 [DefaultDispatcher-worker-1 @request#2] INFO io.ktor.test -- 200 OK: GET - / in 137ms
IPv6 addresses are not supported in NettyConnectionPoint and CIOConnectionPoint
If you access the server through the IPv6 address, it gives an error
exception: java.lang.NumberFormatException: For input string: "ff:ff:ff:ff:ff:ff:ff]:8080"
at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
at java.base/java.lang.Integer.parseInt(Integer.java:668)
at java.base/java.lang.Integer.parseInt(Integer.java:786)
at io.ktor.server.netty.http1.NettyConnectionPoint.getServerPort(NettyConnectionPoint.kt:55)
at io.ktor.server.routing.HostRouteSelector.evaluate(HostsRoutingBuilder.kt:111)
at io.ktor.server.routing.RoutingResolveContext.handleRoute(RoutingResolveContext.kt:97)
at io.ktor.server.routing.RoutingResolveContext.handleRoute(RoutingResolveContext.kt:157)
at io.ktor.server.routing.RoutingResolveContext.resolve(RoutingResolveContext.kt:82)
at io.ktor.server.routing.Routing.interceptor(Routing.kt:62)
at io.ktor.server.routing.Routing$Plugin$install$1.invokeSuspend(Routing.kt:140)
NettyConnectionPoint contains an error. When trying to get the server port or server host, it splits the string at the first occurrence of the ":" character. But you can access the server using the IPv6 address, so it's better to cut off by the last occurrence of the ":" character (substringAfterLast)
NettyConnectionPoint.kt error place:
override val serverPort: Int
get() = request.headers().get(HttpHeaders.Host)
?.substringAfter(":", defaultPort.toString())
?.toInt()
?: localPort
CIO: File upload fails with `NumberFormatException` when uploading file larger than INT_MAX bytes (~2.1 GiB) since 2.3.0
Tried to upload a large-ish file after recently trying out CIO, now faced with this problem on a 12 GiB file:
an error has happened: For input string: "12255718583"
java.lang.NumberFormatException: For input string: "12255718583"
at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.base/java.lang.Integer.parseInt(Integer.java:652)
at java.base/java.lang.Integer.parseInt(Integer.java:770)
at io.ktor.server.cio.CIOApplicationEngine.hasBody(CIOApplicationEngine.kt:153)
at io.ktor.server.cio.CIOApplicationEngine.access$hasBody(CIOApplicationEngine.kt:24)
at io.ktor.server.cio.CIOApplicationEngine$addHandlerForExpectedHeader$2.invokeSuspend(CIOApplicationEngine.kt:135)
at io.ktor.server.cio.CIOApplicationEngine$addHandlerForExpectedHeader$2.invoke(CIOApplicationEngine.kt)
at io.ktor.server.cio.CIOApplicationEngine$addHandlerForExpectedHeader$2.invoke(CIOApplicationEngine.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.request.ApplicationReceiveFunctionsKt.receiveNullable(ApplicationReceiveFunctions.kt:103)
at io.sebi.ui.AddUploadKt$addUpload$2.invokeSuspend(AddUpload.kt:113)
at io.sebi.ui.AddUploadKt$addUpload$2.invoke(AddUpload.kt)
at io.sebi.ui.AddUploadKt$addUpload$2.invoke(AddUpload.kt)
at io.ktor.server.routing.Route$buildPipeline$1$1.invokeSuspend(Route.kt:116)
at io.ktor.server.routing.Route$buildPipeline$1$1.invoke(Route.kt)
at io.ktor.server.routing.Route$buildPipeline$1$1.invoke(Route.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
at io.ktor.server.routing.Routing.executeResult(Routing.kt:190)
at io.ktor.server.routing.Routing.interceptor(Routing.kt:64)
at io.ktor.server.routing.Routing$Plugin$install$1.invokeSuspend(Routing.kt:140)
at io.ktor.server.routing.Routing$Plugin$install$1.invoke(Routing.kt)
at io.ktor.server.routing.Routing$Plugin$install$1.invoke(Routing.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invokeSuspend(BaseApplicationEngine.kt:124)
at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invoke(BaseApplicationEngine.kt)
at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invoke(BaseApplicationEngine.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.server.application.hooks.CallFailed$install$1$1.invokeSuspend(CommonHooks.kt:45)
at io.ktor.server.application.hooks.CallFailed$install$1$1.invoke(CommonHooks.kt)
at io.ktor.server.application.hooks.CallFailed$install$1$1.invoke(CommonHooks.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:61)
at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:261)
at io.ktor.server.application.hooks.CallFailed$install$1.invokeSuspend(CommonHooks.kt:44)
at io.ktor.server.application.hooks.CallFailed$install$1.invoke(CommonHooks.kt)
at io.ktor.server.application.hooks.CallFailed$install$1.invoke(CommonHooks.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invokeSuspend(DefaultEnginePipeline.kt:118)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invoke(DefaultEnginePipeline.kt)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invoke(DefaultEnginePipeline.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2.invokeSuspend(CIOApplicationEngine.kt:238)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:111)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:99)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:585)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:802)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:706)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:693)
CallLogging: the plugin completely overrides MDC if at least one entry is configured
To reproduce make a / request to the following server:
embeddedServer(Netty, port = 3000) {
install(CallLogging) {
mdc("hardcoded") {
"1"
}
}
routing {
val r = route("/") {
handle {
call.respondText { "OK" }
}
}
r.intercept(ApplicationCallPipeline.Setup) {
MDC.put("name", "<name>")
withContext(MDCContext()) {
proceed()
}
}
}
}.start(wait = true)
logback.xml:
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>[%X{name}] %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%kvp- %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="STDOUT" />
</root>
</configuration>
Expected: the MDC key name has value <name> in the logs.
Actual: the MDC key name has an empty value in the logs.
SSE plugin: Duplicated "Content-Type: text/event-stream" headers
Responses from SSE server contains duplicate headers:
HTTP/1.1 200 OK
Vary: Origin
Content-Type: text/event-stream <-------
Cache-Control: no-store
Connection: keep-alive
Content-Type: text/event-stream <-------
transfer-encoding: chunked
This causes the following exception in the JS client:
SSEException: Expected Content-Type text/event-stream but was: text/event-stream, text/event-stream
This comes from installing the SSE plugin on the client/server and sending an sse response as follows:
sse {
send("Hello")
}
2.3.9
released 4th March 2024
Client
ContentNegotiation: the plugin appends duplicated MIME type to Accept header
If I add a header("Accept", "application/json") at request builder block, the content negotiation plugin will add another one,then the header became "Accept": "application/json,application/json"
Generator
Project generator website not reporting created projects metrics
Since March, we haven't been receiving project created metrics from start.ktor.io. This is likely caused from the front-end library updates.
Server
Allow to set secure cookie even with http scheme
Please consider removing the following exception in ResponseCookies :
if (item.secure && !secureTransport) {
throw IllegalArgumentException("You should set secure cookie only via secure transport (HTTPS)")
}
When using a reverse proxy we can't set the secure cookie. I know there is the XForwardedHeaderSupport plugin but it is not customizable (https://youtrack.jetbrains.com/issue/KTOR-2657) and not all configurations will match the specification described in XForwardedHeaderSupport.
The web framework should let the developper decide if they want to send a secure cookie or not.
In the meantime, here is a hotfix to disable this exception without using XForwardedHeaderSupport:
intercept(ApplicationCallPipeline.Features) {
val endpoint = call.attributes.computeIfAbsent(MutableOriginConnectionPointKey) {
MutableOriginConnectionPoint(call.request.origin)
}
endpoint.scheme = "https"
}
2.3.8
released 31st January 2024
Client
WebSockets: Confusing error message when server doesn't respond with Upgrade
Current error description:
io.ktor.client.call.NoTransformationFoundException: Expected response body of the type 'class io.ktor.client.plugins.websocket.DefaultClientWebSocketSession' but was 'class io.ktor.utils.io.ByteBufferChannel'
In response from `https://example.com`
Response status `500 Server Error`
Response header `ContentType: text/xml; charset=UTF-8`
Request header `Accept: application/xml`
Expected error description:
ServerResponseException with the 500 error and ideally the textual response
CIO: "getSubjectAlternativeNames(...) must not be null" error on Android when using CA without SAN since 2.3.5
When using ktor 2.3.5 client with CIO engine on an Android with minSdk of 25, ktor fails in verifyHostnameInCertificate due to: java.lang.NullPointerException: getSubjectAlternativeNames(...) must not be null. This is for a domain which has a custom certificate authority. The custom ca is added to the Android project as a raw resource according to the Android network security configuration, so we have:
- Certificate in
./app/src/main/res/raw/cert.pem AndroidManifest.xmlspecifyingandroid:networkSecurityConfig="@xml/network_security_config"network_security_config.xmlwith following contents:
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<base-config cleartextTrafficPermitted="false">
<trust-anchors>
<certificates src="@raw/cert" />
<certificates src="system" />
</trust-anchors>
</base-config>
</network-security-config>
The CIO engine X509TrustManager documentation says it should default to system, so I assume the above is all that is needed to support a self-signed certificate authority.
When using ktor 2.3.4, the TLS certificate is loaded correctly and the network connection to the domain is opened, so this would appear to be an issue with 2.3.5.
Here's the full stack trace:
10-19 10:14:22.651 11972 11972 E AndroidRuntime: FATAL EXCEPTION: main
19 10:14:22.651 11972 11972 E AndroidRuntime: Process: com.foo.bar, PID: 11972
-19 10:14:22.651 11972 11972 E AndroidRuntime: java.lang.NullPointerException: getSubjectAlternativeNames(...) must not be null
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at io.ktor.network.tls.HostnameUtilsKt.hosts(HostnameUtils.kt:92)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at io.ktor.network.tls.HostnameUtilsKt.verifyHostnameInCertificate(HostnameUtils.kt:19)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at
io.ktor.network.tls.TLSClientHandshake.handleCertificatesAndKeys(TLSClientHandshake.kt:245)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at
io.ktor.network.tls.TLSClientHandshake.negotiate(TLSClientHandshake.kt:166)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at io.ktor.network.tls.TLSClientHandshake$negotiate$1.invokeSuspend(TLSClientHandshake.kt)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at
kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at
kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at
kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
10-19 10:14:22.651 11972 11972 E AndroidRuntime: Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@3b0a279, Dispatchers.Main.immediate]
EDIT: I've attached the public CA file in question.
`URLBuilder` crashes on React Native platforms
On React Native based platforms hasNodeApi will return false and so the platform will end up as BROWSER, but window.location will be undefined, so this code crashes when it tries to retrieve the origin: https://github.com/ktorio/ktor/blob/3.0.0-beta-1/ktor-http/js/src/io/ktor/http/URLBuilderJs.kt#L21
Logging plugin blocks response body streaming when level is BODY
When using the logging plugin (io.ktor.client.plugins.logging.Logging) it causes a block on request streaming, but only when the level is set to LogLevel.BODY. It seems it wants to log it all at the end- instead of as it comes in.
val client = HttpClient {
install(Logging) {
level = LogLevel.BODY
logger = Logger.DEFAULT
}
}
launch {
client.preparePost("...") {
val channel = it.bodyAsChannel()
while (!isClosedForRead) {
awaitContent() // will not return until the request is complete
}
}
}
This is a big issue for any long running streaming endpoints- as you aren't able to disperse the data in real time.
This issue also persists when using MockEngine- which should allow for easier debugging:
val channel = ByteChannel(autoFlush = true)
val mockEngine = MockEngine {
respond(
content = channel,
status = HttpStatusCode.OK
)
}
val client = HttpClient(mockEngine) {
install(Logging) {
level = LogLevel.BODY
logger = Logger.DEFAULT
}
}
val content = channelFlow {
launch {
client.preparePost("...") {
val channel = it.bodyAsChannel()
while (!isClosedForRead) {
awaitContent()
send(readUTF8Line())
}
}
}
}
channel.writeFully("Hello world!\n")
withTimeout(5_000) { // the bug will cause this to timeout
content.collect {
channel.close()
}
}
HttpCache: NumberFormatException for cache-control with max age more than Int.MAX_VALUE
When a valid cache control header like this is passed from a server to the ktor client, an unhandled NumberFormatException occurs
cache-control: max-age=3155695200, public
WebSocket doesn't get terminated when runBlocking is used
I want to start multiple web socket clients that send some requests and then close.
For some reason it gets stuck while closing the socket and doesn't terminate.
If webSocketRaw used instead, it terminates as expected
Example to reproduce:
package com.example
import io.ktor.client.*
import io.ktor.client.engine.cio.*
import io.ktor.client.plugins.websocket.*
import io.ktor.client.plugins.websocket.WebSockets
import io.ktor.client.plugins.websocket.cio.*
import io.ktor.http.*
import io.ktor.server.application.*
import io.ktor.server.engine.*
import io.ktor.server.routing.*
import io.ktor.server.websocket.*
import io.ktor.websocket.*
import kotlinx.coroutines.CoroutineStart
import kotlinx.coroutines.channels.consumeEach
import kotlinx.coroutines.joinAll
import kotlinx.coroutines.launch
import kotlinx.coroutines.runBlocking
import java.time.Duration
fun main() {
val server = embeddedServer(io.ktor.server.cio.CIO, port = 8080) {
install(io.ktor.server.websocket.WebSockets) {
pingPeriod = Duration.ofSeconds(15)
timeout = Duration.ofSeconds(15)
maxFrameSize = Long.MAX_VALUE
masking = false
}
routing {
webSocket("/echo") {
incoming.consumeEach { frame ->
if (frame is Frame.Text) {
val receivedText = frame.readText()
send(Frame.Text("You sent: $receivedText"))
}
}
}
}
}.start()
val CLIENTS_COUNT = 10
val REQUEST_COUNT = 2000
runBlocking {
val jobs = (0 until CLIENTS_COUNT).map { clientId ->
launch(start = CoroutineStart.LAZY) {
HttpClient(CIO) {
install(WebSockets) {
pingInterval = 20_000
}
}.use { client ->
client.webSocket(method = HttpMethod.Get, host = "127.0.0.1", port = 8080, path = "/echo") {
for (requestId in 0..REQUEST_COUNT) {
send(Frame.Text("$clientId-$requestId"))
}
}
}
}
}
jobs.forEach { it.start() }
jobs.joinAll()
}
println("Stopping")
server.stop(0)
}
Core
"ReferenceError: 'self' is not defined" when using URLBuilder in a custom JS engine
So I'm playing around with https://github.com/cashapp/zipline
I'm running Ktor in JS.
Zipline runs the compiled JS in a custom QuickJS engine.
In this case, the PlatformUtilsJs.kt code is picking up that we're a Platform.Browser because process.versions.node does not exist.
Then URLBuilderJs.kt code is trying to fetch the origin from self.location.origin which doesn't exist and errors out.
Here's my stacktrace
E/Zipline: ReferenceError: 'self' is not defined
at <anonymous> (mnt/agent/work/8d547b974a7be21f/ktor-http/js/src/io/ktor/http/URLBuilderJs.kt:22)
at get_origin (mnt/agent/work/8d547b974a7be21f/ktor-http/js/src/io/ktor/http/URLBuilderJs.kt:17)
at Companion_6 (mnt/agent/work/8d547b974a7be21f/ktor-http/common/src/io/ktor/http/URLBuilder.kt:116)
at Companion_getInstance_9 (ktor-ktor-http.js)
at URLBuilder (mnt/agent/work/8d547b974a7be21f/ktor-http/common/src/io/ktor/http/URLBuilder.kt:25)
at HttpRequestBuilder (mnt/agent/work/8d547b974a7be21f/ktor-client/ktor-client-core/common/src/io/ktor/client/request/HttpRequest.kt:65)
I'm invoking this snippet before the URL code as a workaround instead for now
js(
"""
globalThis.self = {
location: {},
};
"""
)
Thanks.
Docs
Documentation Restructuring: Getting started with Ktor Server
Currently, we have a 'Creating a new Ktor project' tutorial which shows how to create a new project with the IDE plugin. The project structure is examined in each of the following tutorials. In the new tutorial structure, we need to:
- show how to create a project using the generator
- examine the project structure
- show additional tasks to attempt
New content is available here
Generator
Gradle wrapper `gradlew` generated by wizard is not executable by default
To reproduce: Generate project using https://start.ktor.io/.
Try to run via ./gradlew run. See zsh: permission denied: ./gradlew.
Workaround: chmod +x gradlew.
Wizard should set correct permissions for the file when generating and zipping the project.
Server
"KeyStoreException: JKS not found" exception on Android when configuring secure connection
To reproduce, run the following code on an Android emulator:
val keyStoreFile = File("build/keystore.jks")
val keyStore = buildKeyStore {
certificate("sampleAlias") {
password = "foobar"
domains = listOf("127.0.0.1", "0.0.0.0", "localhost")
}
}
keyStore.saveToFile(keyStoreFile, "123456")
val environment = applicationEngineEnvironment {
log = LoggerFactory.getLogger("ktor.application")
connector {
port = 8080
}
sslConnector(
keyStore = keyStore,
keyAlias = "sampleAlias",
keyStorePassword = { "123456".toCharArray() },
privateKeyPassword = { "foobar".toCharArray() }) {
port = 8443
keyStorePath = keyStoreFile
}
module {}
}
embeddedServer(Netty, environment).start(wait = true)
As a result the KeyStoreException is thrown:
FATAL EXCEPTION: DefaultDispatcher-worker-1
Process: com.example.ktor_android, PID: 8259
java.security.KeyStoreException: JKS not found
at java.security.KeyStore.getInstance(KeyStore.java:904)
at io.ktor.network.tls.certificates.KeyStoreBuilder.build$ktor_network_tls_certificates(builders.kt:160)
at io.ktor.network.tls.certificates.BuildersKt.buildKeyStore(builders.kt:176)
at com.example.ktor_android.MainActivity$onCreate$1.invokeSuspend(MainActivity.kt:31)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:100)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@5e4a1a6, Dispatchers.IO]
Caused by: java.security.NoSuchAlgorithmException: JKS KeyStore not available
at sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
at java.security.Security.getImpl(Security.java:628)
at java.security.KeyStore.getInstance(KeyStore.java:901)
... 11 more
MDC diagnostic value is changed during logging of the request
I have a simple call logging use case. When request comes in, we generate a requestId (uuid), then log this requestId for all logging messages that occurred during the processing of this request. This is used for easier debugging of issues, as we return this requestId in the response. Now, the problem is that I seem to get multiple different requestIds for a single call. Below in the sample log you can see there are three different requestId (contained in <>). I would expect all the logs have the same requestId. Am I misunderstanding how this should work, or is this a genuine bug?
I'm running this on Ktor 2.3.5, using Koin 3.5.1 and log4j2 2.21.1 in a kotlin multiplatform project.
Sample log:
2023-11-30 10:03:33,574 INFO [Application] (eventLoopGroupProxy-4-1 @call-handler#6) <21e7bfa3-06de-4fc2-a1cd-85bda022dae4> GET USER
2023-11-30 10:03:33,775 INFO [org.kto.database] (eventLoopGroupProxy-4-1 @call-handler#6) <21e7bfa3-06de-4fc2-a1cd-85bda022dae4> Connected to jdbc:postgresql://localhost:5432/test, productName: PostgreSQL, productVersion: 16.0 (Debian 16.0-1.pgdg120+1), logger: org.ktorm.logging.Slf4jLoggerAdapter@33988138, dialect: org.ktorm.support.postgresql.PostgreSqlDialect@2df38877
2023-11-30 10:37:44,912 INFO [UserService] (eventLoopGroupProxy-4-1 @call-handler#6) <21e7bfa3-06de-4fc2-a1cd-85bda022dae4> IN SERVICE
2023-11-30 10:03:33,999 DEBUG [org.kto.database] (eventLoopGroupProxy-4-1 @call-handler#6) <21e7bfa3-06de-4fc2-a1cd-85bda022dae4> SQL: select <actual_select>
2023-11-30 10:03:33,999 DEBUG [org.kto.database] (eventLoopGroupProxy-4-1 @call-handler#6) <21e7bfa3-06de-4fc2-a1cd-85bda022dae4> Parameters: <select_params>
2023-11-30 10:03:34,032 DEBUG [org.kto.database] (eventLoopGroupProxy-4-1 @call-handler#6) <21e7bfa3-06de-4fc2-a1cd-85bda022dae4> Results: 0
2023-11-30 10:03:34,048 ERROR [Application] (eventLoopGroupProxy-4-1 @call-handler#6) <526e4baa-694c-44a7-8fd0-fce8a9de3cd2> [/users/10a9c534-30ea-4e2f-bd48-b151a7597991] Unknown exception occurred.Reference requestId: '526e4baa-694c-44a7-8fd0-fce8a9de3cd2'. Message: 'null'
<packages...>.server.exception.UserNotFoundException: null
at <packages...>.server.service.UserService.getUser(UserService.kt:19) ~[main/:?]
at <packages...>.server.routing.UserRouteKt$userRoutes$1$2.invokeSuspend(UserRoute.kt:30) ~[main/:?]
2023-11-30 10:03:34,095 INFO [Application] (eventLoopGroupProxy-4-1 @call-handler#6) <fb918a74-af4b-42f9-b747-caafd368577a> Status: 500 Internal Server Error, HTTP method: GET, User agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36
In the logging example above, it can be seen that the same request had three different requestIds. The correct behaviour should be, that all logs should have the same requestId logged. The CallLogging logs have a different requestId from StatusPages exception handling logs and from the actual route logs...
The code:
// Call logging configuration
install(CallLogging) {
level = Level.INFO
mdc("requestId") {
UUID.randomUUID().toString()
}
format { call ->
val status = call.response.status()
val httpMethod = call.request.httpMethod.value
val userAgent = call.request.headers["User-Agent"]
"Status: $status, HTTP method: $httpMethod, User agent: $userAgent"
}
}
...
// Exception handling
install(StatusPages) {
exception<Throwable> { call, cause ->
val statusCode = HttpStatusCode.InternalServerError
val requestId = MDC.get("requestId")
val additionalMessage = "Reference requestId: '${requestId}'. Message: '${cause.message}'"
val err = Error.UNKNOWN_EXCEPTION.withAdditionalMessage(additionalMessage)
call.application.log.error("[${call.request.path()}] Unknown exception occurred. $additionalMessage", cause)
call.respond(statusCode, ErrorResponse(err.errorCode, err.errorMessage, err.additionalMessage))
}
}
...
// Route config
get("/users/{userId}") {
call.application.log.info("GET USER")
val user = userService.getUser(UUID.fromString(call.parameters["userId"]))
call.respond(HttpStatusCode.OK, user)
}
...
// UserService
private val logger: Logger = LoggerFactory.getLogger(this.javaClass.simpleName)
fun getUser(id: UUID): UserResponse {
logger.info("IN SERVICE")
val user = userRepository.findById(id) ?: throw UserNotFoundException(id)
return mapUserToUserResponse(user)
}
// Log4j2 PatternLayout
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c{3.}] (%t) <%X{requestId}> %msg%n" />
Thank you, Jan
ContentNegotiation: Adding charset to content type of JacksonConverter breaks request matching
When trying to get the ContentNegotiation to specify the charset on JSON responses like so:
install(ContentNegotiation) {
jackson(contentType = ContentType.Application.Json.withCharset(Charsets.UTF_8))
}
the matching on ingoing requests breaks.
When looking at RequestConverter this is no surprise, since it strips the charset from the requestContentType and then tries to match the full content type provided at registration.
val requestContentType = try {
call.request.contentType().withoutParameters()
}
...
if (!requestContentType.match(registration.contentType)) {
LOGGER.trace(
"Skipping content converter for request type ${receiveType.type} because " +
"content type $requestContentType does not match ${registration.contentType}"
)
return null
}
I guess the charset should also be stripped from the registration content type or there should be another way to specify the charset parameter for responses which does not affect the matching for requests.
High Native Server Memory Usage
I have a "hello, world" Ktor native server (Kotlin/Native on Linux) and have noticed very high memory usage under heavy load. While memory usage starts around 30MB, it climbs to around 9GB when the server is saturated with requests. Memory does fall back to around 125MB after requests have ceased. I've tried different allocators and options but don't see much difference.
Repo for reproducing:
https://github.com/jamesward/ktor-native-server/tree/mem_repro
ApacheBench for request saturation:
ab -c 8 -n 1000000 http://localhost:8080/
I know it's an apples-to-oranges comparison, but running the same thing on the JVM doesn't seem to cause any memory spiking.
I'm not sure if there are optimizations in Ktor for this use case or if it is all on Kotlin/Native.
Server ContentNegotiation no longer allows multiple decoders for one Content-Type
In Ktor Server 2.1.3, it was possible to have multiple ContentNegotiation "codecs" installed for one Content-Type. During request decoding, the codecs were tried sequentially and the first non-null result was used. In Ktor 2.2.1, this no longer works - only the first codec is tried. This causes issues for example when using org.json and Kotlinx.serialization in parallel (I'm using a custom shim converter that allows me to directly receive JSONObjects).
I suppose the problem appeared in commit https://github.com/ktorio/ktor/commit/be329698362b9c12ca43b5969bdbbcbdb3496081. I don't know the exact mechanism at play here, but I think there is a subtle logic change in the reimplemented convertBody function or its caller.
This demo reproduces the problem:
- Application.kt:
package com.example
import io.ktor.http.ContentType
import io.ktor.server.application.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import io.ktor.server.plugins.contentnegotiation.ContentNegotiation
import io.ktor.server.request.receive
import io.ktor.server.response.respondText
import io.ktor.server.routing.post
import io.ktor.server.routing.routing
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0", module = Application::module)
.start(wait = true)
}
fun Application.module() {
install(ContentNegotiation) {
cvtA(ContentType.Application.Json)
cvtB(ContentType.Application.Json)
}
routing {
post("/") {
val request = call.receive<CvtBResult>()
call.respondText("ECHO: " + request.data)
}
}
}
- CvtA.kt:
package com.example
import io.ktor.http.ContentType
import io.ktor.http.content.OutgoingContent
import io.ktor.http.content.TextContent
import io.ktor.http.withCharsetIfNeeded
import io.ktor.serialization.Configuration
import io.ktor.serialization.ContentConverter
import io.ktor.util.reflect.TypeInfo
import io.ktor.utils.io.ByteReadChannel
import io.ktor.utils.io.charsets.Charset
import io.ktor.utils.io.jvm.javaio.toInputStream
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
data class CvtAResult(val data: String)
class CvtA : ContentConverter {
override suspend fun deserialize(charset: Charset, typeInfo: TypeInfo, content: ByteReadChannel): Any? {
return if (typeInfo.type == CvtAResult::class) {
withContext(Dispatchers.IO) {
val reader = content.toInputStream().reader(charset)
CvtAResult(reader.readText())
}
} else {
null
}
}
override suspend fun serializeNullable(
contentType: ContentType,
charset: Charset,
typeInfo: TypeInfo,
value: Any?
): OutgoingContent? {
return if (value != null && typeInfo.type == CvtAResult::class) {
TextContent((value as CvtAResult).data, contentType.withCharsetIfNeeded(charset))
} else {
null
}
}
}
/**
* Register a converter for org.JSON objects.
*/
fun Configuration.cvtA(
contentType: ContentType = ContentType.Application.Json,
) {
register(contentType, CvtA())
}
- CvtB:kt:
package com.example
import io.ktor.http.ContentType
import io.ktor.http.content.OutgoingContent
import io.ktor.http.content.TextContent
import io.ktor.http.withCharsetIfNeeded
import io.ktor.serialization.Configuration
import io.ktor.serialization.ContentConverter
import io.ktor.util.reflect.TypeInfo
import io.ktor.utils.io.ByteReadChannel
import io.ktor.utils.io.charsets.Charset
import io.ktor.utils.io.jvm.javaio.toInputStream
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
data class CvtBResult(val data: String)
class CvtB : ContentConverter {
override suspend fun deserialize(charset: Charset, typeInfo: TypeInfo, content: ByteReadChannel): Any? {
return if (typeInfo.type == CvtBResult::class) {
withContext(Dispatchers.IO) {
val reader = content.toInputStream().reader(charset)
CvtBResult(reader.readText())
}
} else {
null
}
}
override suspend fun serializeNullable(
contentType: ContentType,
charset: Charset,
typeInfo: TypeInfo,
value: Any?
): OutgoingContent? {
return if (value != null && typeInfo.type == CvtBResult::class) {
TextContent((value as CvtBResult).data, contentType.withCharsetIfNeeded(charset))
} else {
null
}
}
}
/**
* Register a converter for org.JSON objects.
*/
fun Configuration.cvtB(
contentType: ContentType = ContentType.Application.Json,
) {
register(contentType, CvtB())
}
Test request can be made via curl:
curl -i http://localhost:8080/ -H "Content-Type: application/json" --data '{}'
I expected the following output (behaviour of Ktor 2.1.3):
HTTP/1.1 200 OK
Content-Length: 8
Content-Type: text/plain; charset=UTF-8
ECHO: {}
However, with Ktor 2.2.1 this is printed:
HTTP/1.1 415 Unsupported Media Type
Content-Length: 0
As for the parallel use of org.json and kotlinx.serialization: I've previously used org.json only and then introduced kotlinx.serialization for handling data with static schema. I've now noticed that kotlinx.serialization.json provides JsonElement which appears to be able to handle data without a fixed schema. I might therefore switch away from using org.json entirely and thus this issue wouldn't affect me anymore.
{...} (tailcard) does not match URLs ending with '/'
Example route:
get("/api/subtree/{...}") {
...
}
Expected:
- /api/subtree - do not match (according to KTOR-372 logic)
- /api/subtree/ - match
- /api/subtree/lvl1 - match
- /api/subtree/lvl1/ - match
- /api/subtree/lvl1/lvl2 - match
- /api/subtree/lvl1/lvl2/ - match
Current behavior:
- /api/subtree - match
- /api/subtree/ - do not match
- /api/subtree/lvl1 - match
- /api/subtree/lvl1/ - do not match
- /api/subtree/lvl1/lvl2 - match
- /api/subtree/lvl1/lvl2/ - do not match
I read about using two routes (with and without trailing '/' to cover both cases) or implementing redirect logic.
Those will work for static routes, but how to solve this for wildcard subtree route?
CORS: `allowHost` without the second argument doesn't allow the secure host
I would like to request the addition of HTTPS to the schema argument of allowHost in CORSConfig. Currently, when only HTTP is used, it works correctly in a local environment even without setting the schema argument. However, when deployed to an HTTPS development environment, the absence of a schema argument causes it not to function, leading to time-consuming troubleshooting.
RequestConnectionPoint should implement toString()
Instances of io.ktor.http.RequestConnectionPoint are sometimes used in logs, for example here:
This produces logs like this this Netty for example:
12:13:39.734 TRACE - Respond forbidden /: origin doesn't match io.ktor.server.netty.http1.NettyConnectionPoint@7a5bf5c3
Such log does not help with debugging CORS, I don't know if there are more instances of origin used in logs.
Other
CIO: Unable to perform WebSocket upgrade when Content-Type header is sent in the request
Hello, I have a web server that is built using Ktor, and is deployed on Heroku, In this server I configured WebSockets and created the following route:
fun Application.configureSockets() {
val json by inject<Json>()
install(WebSockets) {
pingPeriod = Duration.ofSeconds(15)
timeout = Duration.ofSeconds(15)
maxFrameSize = Long.MAX_VALUE
masking = false
contentConverter = KotlinxWebsocketSerializationConverter(json)
}
}
fun Route.getEnrolledStudentSocketRoute() {
val services by inject<ClassServices>()
webSocket(application.href(EndPoint.Socket.Class.EnrolledStudents())) {
val classId = call.parameters["classId"] ?: return@webSocket close(CloseReason(CloseReason.Codes.CANNOT_ACCEPT, "class id is missing"))
services.getEnrolledStudents(classId).collectLatest { students ->
sendSerialized(EnrolledStudentsDto(classId, students.map(Student::asDto)))
}
}
}
From the client side, I have an Android application with Ktor Client, which is configured to use websockets, I have this request:
internal fun HttpClientConfig<*>.configureWebSockets(json: Json) {
WebSockets {
contentConverter = KotlinxWebsocketSerializationConverter(json)
}
}
override suspend fun getEnrolledStudents(classId: String) = channelFlow {
val url = client.href(TeacherStudentPackagesApi.WebSocket.EnrolledStudents(classId))
client.wss(url) {
incoming.consumeEach {
this@channelFlow.send(receiveDeserialized())
}
}
}
However, when I try to establish the connection, the server always throw this error:
java.lang.IllegalStateException: Unable to perform upgrade as it is not requested by the client: request should have Upgrade and Connection headers filled properly
at io.ktor.server.cio.CIOApplicationResponse.respondOutgoingContent(CIOApplicationResponse.kt:114)
at io.ktor.server.engine.BaseApplicationResponse$Companion$setupSendPipeline$1.invokeSuspend(BaseApplicationResponse.kt:317)
at io.ktor.server.engine.BaseApplicationResponse$Companion$setupSendPipeline$1.invoke(BaseApplicationResponse.kt)
at io.ktor.server.engine.BaseApplicationResponse$Companion$setupSendPipeline$1.invoke(BaseApplicationResponse.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(SuspendFunctionGun.kt:88)
at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invokeSuspend(DefaultTransform.kt:29)
at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.websocket.RoutingKt.respondWebSocketRaw(Routing.kt:293)
at io.ktor.server.websocket.RoutingKt.access$respondWebSocketRaw(Routing.kt:1)
at io.ktor.server.websocket.RoutingKt$webSocketRaw$2$1$1$1.invokeSuspend(Routing.kt:105)
at io.ktor.server.websocket.RoutingKt$webSocketRaw$2$1$1$1.invoke(Routing.kt)
at io.ktor.server.websocket.RoutingKt$webSocketRaw$2$1$1$1.invoke(Routing.kt)
at io.ktor.server.routing.Route$buildPipeline$1$1.invokeSuspend(Route.kt:116)
at io.ktor.server.routing.Route$buildPipeline$1$1.invoke(Route.kt)
at io.ktor.server.routing.Route$buildPipeline$1$1.invoke(Route.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
at io.ktor.server.routing.Routing.executeResult(Routing.kt:190)
at io.ktor.server.routing.Routing.interceptor(Routing.kt:64)
at io.ktor.server.routing.Routing$Plugin$install$1.invokeSuspend(Routing.kt:140)
at io.ktor.server.routing.Routing$Plugin$install$1.invoke(Routing.kt)
at io.ktor.server.routing.Routing$Plugin$install$1.invoke(Routing.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invokeSuspend(BaseApplicationEngine.kt:124)
at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invoke(BaseApplicationEngine.kt)
at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invoke(BaseApplicationEngine.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.server.application.hooks.CallFailed$install$1$1.invokeSuspend(CommonHooks.kt:45)
at io.ktor.server.application.hooks.CallFailed$install$1$1.invoke(CommonHooks.kt)
at io.ktor.server.application.hooks.CallFailed$install$1$1.invoke(CommonHooks.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:78)
at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:264)
at io.ktor.server.application.hooks.CallFailed$install$1.invokeSuspend(CommonHooks.kt:44)
at io.ktor.server.application.hooks.CallFailed$install$1.invoke(CommonHooks.kt)
at io.ktor.server.application.hooks.CallFailed$install$1.invoke(CommonHooks.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invokeSuspend(DefaultEnginePipeline.kt:123)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invoke(DefaultEnginePipeline.kt)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invoke(DefaultEnginePipeline.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2.invokeSuspend(CIOApplicationEngine.kt:239)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
The request log from the client side:
REQUEST: wss://<BASE_URL>/ws/class/students?classId=default
METHOD: HttpMethod(value=GET)
COMMON HEADERS
-> Accept: application/json
-> Accept-Charset: UTF-8
-> Content-Type: application/json
CONTENT HEADERS
-> Connection: upgrade
-> Sec-WebSocket-Key: YTA4OWRkNDQ1YjU0YmVmYQ==
-> Sec-WebSocket-Version: 13
-> Upgrade: websocket
BODY Content-Type: null
BODY START
BODY END
That is weird, other clients such as postman can establish a successful connection, but the Android Ktor client does not. I am using CIO engine on both server and client, I have tried other JVM engines on my client, but all didn't work. From the above request log, found that `Upgrade` and `Connection` headers are properly filled, so I can't understand where is the problem exactly, It seems like a bug for me, I also tried some 2.3.x older versions, the error still arises.
2.3.7
released 7th December 2023
Docs
Broken link in "Logging"
Documentation Restructuring: Welcome page
Current State
Currently, the entry point of the docs is the 'Welcome' page. For a starting page, nothing really stands out apart from the Ktor logo. The 'featured topics' look like second navigation where a summary is provided for each topic on mouse hover. The summaries are not consistent in style and in some cases show random content (e.g. 'Custom plugins' shows 'code example: custom-plugin').
Proposed Solution
Restructure the page using cards and groups supplied by Writerside's starting pages. Along with this, the following changes can be made:
- change the title to "Ktor Documentation" (if keeping the logo, make it smaller)
- keep the introduction, but refine it to add more clarity and flow
- below the introduction, show buttons leading to the Getting Started tutorials, such as:
{width=70%}
- the rest of the topics can be presented as grouped links with tabs for Client and Server. For example:
- "First steps" to display links for the topics under "Configuring a server" and "Setting up a client" or core concepts, such as "Plugins"
2.3.6
released 7th November 2023
Client
Darwin: EOFException when sending multipart data using Ktor 2.3.4
After bumping ktor client version from 2.3.3 to 2.3.4 we are getting a "Optional(io.ktor.utils.io.errors.EOFException: Premature end of stream: expected 467 bytes)) error" on iOS (works fine on Android) when sending a form with binary data.
The code simplified looks like this (image is a byte array):
val formData = formData {
append(
key = "image",
value = image,
headers = Headers.build {
append(HttpHeaders.ContentType, "image/jpeg")
append(HttpHeaders.ContentDisposition, "filename=image.jpeg")
},
)
}
return httpClient.submitFormWithBinaryData(url, formData) {}.body()
We didn't change anything on our side and downgrading to 2.3.3 fixes the issue, so it seems to be related to the update.
AndroidClientEngine cannot handle content length that exceeds Int range
In execute() method in AndroidClientEngine class, following line makes it cannot handle content length that exceeds integer range.
contentLength?.let { setFixedLengthStreamingMode(it.toInt()) } ?: setChunkedStreamingMode(0)
Since there is setFixedLengthStreamingMode(long), removing toInt() call may fix the issue.
Darwin: Even a coroutine Job is canceled network load keeps high
I guess the problem is caused by DarwinSession.close:
override fun close() {
if (!closed.compareAndSet(false, true)) return
session.finishTasksAndInvalidate()
}
In Apple's document,
This method returns immediately without waiting for tasks to finish. Once a session is invalidated, new tasks cannot be created in the session, but existing tasks continue until completion . After the last task finishes and the session makes the last delegate call related to those tasks, the session calls the URLSession:didBecomeInvalidWithError: method on its delegate, then breaks references to the delegate and callback objects. After invalidation, session objects cannot be reused.
To cancel all outstanding tasks, call invalidateAndCancel instead.
So, in close function, invoking invalidateAndCancel is better than finishTasksAndInvalidate
Ktor JS client unconfigurable logging in node
Many ktor client plugins contain their own LOGGER, e.g
internal val LOGGER = KtorSimpleLogger("io.ktor.client.plugins.websocket.WebSockets")
Which can't be configured.
Implementation of KtorSimpleLoggerJs.kt
@Suppress("FunctionName")
public actual fun KtorSimpleLogger(name: String): Logger = object : Logger {
override fun error(message: String) {
console.error(message)
}
override fun error(message: String, cause: Throwable) {
console.error("$message, cause: $cause")
}
override fun warn(message: String) {
console.warn(message)
}
override fun warn(message: String, cause: Throwable) {
console.warn("$message, cause: $cause")
}
override fun info(message: String) {
console.info(message)
}
override fun info(message: String, cause: Throwable) {
console.info("$message, cause: $cause")
}
override fun debug(message: String) {
console.debug("DEBUG: $message")
}
override fun debug(message: String, cause: Throwable) {
console.debug("DEBUG: $message, cause: $cause")
}
override fun trace(message: String) {
console.debug("TRACE: $message")
}
override fun trace(message: String, cause: Throwable) {
console.debug("TRACE: $message, cause: $cause")
}
}
It floods logs with unnecessary and useless messages on Node as console.debug is just an alias for console.log
TRACE: Sending WebSocket request [object Object]
This logger should be configurable with Logging plugin.
"Server sent a subprotocol but none was requested" when using Node WebSockets
The following code fails with "Server sent a subprotocol but none was requested":
val ktorClient = HttpClient(Js) {
install(WebSockets)
}
val url = Url("wss://apollo-fullstack-tutorial.herokuapp.com/graphql")
val connection = ktorClient.request<DefaultClientWebSocketSession>(url) {
headers {
append("Sec-WebSocket-Protocol", "graphql-ws")
}
}
This seems to happen because protocols is never passed to the WebSocket constructor (here)
Client unable to make subsequent requests after the network disconnection and connection when ResponseObserver is installed
Description:
When using the Ktor HTTP client library to download files, an issue has been observed where the client fails to recover after an interrupted download caused by a loss of internet connection. This behavior is particularly evident when the provided code snippet is used to perform the download.
Steps to Reproduce:
- Establish a working internet connection.
- Execute the provided code snippet for making an HTTP request to download a file using the Ktor HTTP client.
- During the file download process, intentionally disrupt the internet connection.
- Allow the client to handle the connection loss and observe its behavior.
Expected Behavior: Upon detecting a loss of connectivity during the file download process, the Ktor HTTP client should throw an appropriate exception (e.g., a network-related exception) to indicate the connection failure. Subsequent attempts to make new HTTP requests using the client, after restoring the internet connection, should work as expected. The client should no longer be stuck in an unrecoverable state and should be capable of handling further requests without hindrance.
Actual Behavior: Currently, when the internet connection is disrupted during the file download process, the Ktor HTTP client appears to enter an unrecoverable state. Subsequent attempts to use the client to make additional HTTP requests (even after the internet connection is restored) fail to complete successfully. The client seems to be stuck in a state where it cannot recover from the interrupted download.
Code Snippet:
try {
val httpResponse: HttpResponse = httpClient.get(url) {
onDownload { bytesSentTotal, contentLength ->
// Progress tracking logic can be added here if needed
}
}
val responseBody: ByteArray = httpResponse.body()
file.writeBytes (responseBody)
} catch (e: Exception) {
// Exception handling logic can be added here
}
Exception Stack Trace:
java.net.SocketTimeoutException: timeout at com.android.okhttp.okio.Okio$3.newTimeoutException(Okio.java:214) at com.android.okhttp.okio.AsyncTimeout.exit(AsyncTimeout.java:263) at com.android.okhttp.okio.AsyncTimeout$2.read(AsyncTimeout.java:217) at com.android.okhttp.okio.RealBufferedSource.read(RealBufferedSource.java:51) at com.android.okhttp.internal.http.Http1xStream$FixedLengthSource.read(Http1xStream.java:395) at com.android.okhttp.okio.RealBufferedSource$1.read(RealBufferedSource.java:372) at java.io.BufferedInputStream.fill(BufferedInputStream.java:248) at java.io.BufferedInputStream.read1(BufferedInputStream.java:288) at java.io.BufferedInputStream.read(BufferedInputStream.java:347) at io.ktor.utils.io.jvm.javaio.ReadingKt$toByteReadChannel$1.invokeSuspend(Reading.kt:55) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684) Caused by: java.net.SocketException: Socket closed at java.net.SocketInputStream.read(SocketInputStream.java:188) at java.net.SocketInputStream.read(SocketInputStream.java:143) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.readFromSocket(ConscryptEngineSocket.java:945) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.processDataFromSocket(ConscryptEngineSocket.java:909) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.readUntilDataAvailable(ConscryptEngineSocket.java:824) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.read(ConscryptEngineSocket.java:797) at com.android.okhttp.okio.Okio$2.read(Okio.java:138) at com.android.okhttp.okio.AsyncTimeout$2.read(AsyncTimeout.java:213) at com.android.okhttp.okio.RealBufferedSource.read(RealBufferedSource.java:51) at com.android.okhttp.internal.http.Http1xStream$FixedLengthSource.read(Http1xStream.java:395) at com.android.okhttp.okio.RealBufferedSource$1.read(RealBufferedSource.java:372) at java.io.BufferedInputStream.fill(BufferedInputStream.java:248) at java.io.BufferedInputStream.read1(BufferedInputStream.java:288) at java.io.BufferedInputStream.read(BufferedInputStream.java:347) at io.ktor.utils.io.jvm.javaio.ReadingKt$toByteReadChannel$1.invokeSuspend(Reading.kt:55) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
Note:
The preceding stack trace remains consistent for both the unsuccessful download request and the subsequent requests that follow it.
Environment:
- Ktor version: [2.3.3]
- HttpClientEngine: Android
- Operating System: [Android 11]
WebSockets (CIO): Connection Failure Due to Lowercase 'upgrade' in 'Connection: upgrade' Header
Hello,
I have encountered an issue while trying to establish a WebSocket connection using Ktor. It appears that the 'Connection: upgrade' header sent by Ktor has 'upgrade' in lowercase, which is causing the WebSocket connection to fail.
In my application scenario, I tried CIO, Java, and OkHttp engines and captured packets for analysis. Only the OkHttp engine used the header with capital letters when establishing a connection, that is, "Connection: Upgrade", and then the connection was successful, while the other both engines use lowercase "upgrade", which causes the connection to fail.
By reading the code, I found that Ktor used lowercase "upgrade" here. I believe adjusting the capitalization to 'Connection: Upgrade' may resolve the WebSocket connection issue. I hope this information is helpful and look forward to hearing back on whether this could be addressed in a future release of Ktor.
Thank you.
WinHttp: ArrayIndexOutOfBoundsException when sending WS frame with empty body
It appears that sending a frame with an empty body (e.g. a Frame.Text("")) crashes an internal coroutine that fails incoming channel collections.
The root cause is the following:
Caused by: kotlin.ArrayIndexOutOfBoundsException
at 0 ??? 7ff7ae9c8d62 kfun:kotlin.Throwable#<init>(){} + 98
at 1 ??? 7ff7ae9c2f7f kfun:kotlin.Exception#<init>(){} + 79
at 2 ??? 7ff7ae9c31ef kfun:kotlin.RuntimeException#<init>(){} + 79
at 3 ??? 7ff7ae9c337f kfun:kotlin.IndexOutOfBoundsException#<init>(){} + 79
at 4 ??? 7ff7ae9c3a4f kfun:kotlin.ArrayIndexOutOfBoundsException#<init>(){} + 79
at 5 ??? 7ff7ae9f1028 ThrowArrayIndexOutOfBoundsException + 120
at 6 ??? 7ff7aed892a7 Kotlin_Arrays_getByteArrayAddressOfElement + 23
at 7 ??? 7ff7ae9b8ed1 kfun:kotlinx.cinterop#addressOf__at__kotlinx.cinterop.Pinned<kotlin.ByteArray>(kotlin.Int){}kotlinx.cinterop.CPointer<kotlinx.cinterop.ByteVarOf<kotlin.Byte>> + 177
at 8 ??? 7ff7aed26b9d kfun:io.ktor.client.engine.winhttp.internal.WinHttpWebSocket.sendFrame#internal + 2877
at 9 ??? 7ff7aed255f4 kfun:io.ktor.client.engine.winhttp.internal.WinHttpWebSocket.$sendNextFrameCOROUTINE$0.invokeSuspend#internal + 2580
at 10 ??? 7ff7ae9ce785 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 1029
It turns out the WinHttpWebSocket tries to use buffer.addressOf(0) even when the body (buffer) is empty:
https://github.com/ktorio/ktor/blob/b34415dafca82090e408761788a9b7ffb24788d7/ktor-client/ktor-client-winhttp/windows/src/io/ktor/client/engine/winhttp/internal/WinHttpWebSocket.kt#L201
I think the same issue is probably true for receiving frames with empty bodies:
https://github.com/ktorio/ktor/blob/b34415dafca82090e408761788a9b7ffb24788d7/ktor-client/ktor-client-winhttp/windows/src/io/ktor/client/engine/winhttp/internal/WinHttpWebSocket.kt#L109
Note that it might be hard to notice due to https://youtrack.jetbrains.com/issue/KT-62794 (we need to use printStackTrace() to see the cause).
Reproducer:
fun main() {
runBlocking {
val http = HttpClient(WinHttp) {
install(WebSockets)
}
val session = http.webSocketSession("wss://ws.postman-echo.com/raw")
try {
session.send(Frame.Text(""))
// This fails with ArrayIndexOutOfBoundsException, not because of the received echoed frame,
// but because of the failed send causing the incoming channel to fail (see sendFrame in the stacktrace)
session.incoming.receive()
} finally {
session.close()
}
}
}
Docs
The generator adds the '-jvm' suffix to all dependencies
Generator
The newly generated project is having problems with gradle
I think We have the same bug
{width=70%}
After updating ktor plugin (for gradle) to version 2.3.6, we also updated jackson-core-2.15.2 . But it has some problems with gradle 7.5.1 .
With the ktor plugin 2.3.5, it works fine
Because of this, the newly created project does not build in ide
Gradle Plugin
Outdated Gradle jib plubin does not support application/vnd.oci.image.index.v1+json media type
If you try to use a gcr.io image as a base image for containerization, you will get the following error:
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':jibDockerBuild'.
> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: Tried to pull image manifest for gcr.io/distroless/java17-debian11:debug but failed because: Manifest with tag 'debug' has media type 'application/vnd.oci.image.index.v1+json', but client accepts 'application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.v2+json,application/vnd.docker.distribution.manifest.v1+json,application/vnd.docker.distribution.manifest.list.v2+json'.
Adding the latest version of jib plugin explicitly solves the problem.
How to reproduce:
- generate fresh Ktor project at https://start.ktor.io/ (Ktor 2.3.4 as for now)
- try to
./gradlew -Djib.from.image='gcr.io/distroless/java17-debian11:debug' jibDockerBuildand you will get the above error
Workaround:
Just add jib plugin explicitly.
plugins {
kotlin("jvm") version "1.9.10"
id("io.ktor.plugin") version "2.3.4"
id("com.google.cloud.tools.jib") version "3.3.2" // <<------------
}
Server
CIO: getEngineHeaderValues() returns duplicated values
Description:
When using embeddedCIOServer and geting all headers with call.response.headers.allValues() returned values are not as expected (examples below). When switching to another engine(e.g. Netty) returned values are as expected.
Describe the bug:
Function allValues() uses abstract function getEngineHeaderValues(name: String) from io.ktor.server.cio.CIOApplicationResponse and its implementation in cio-server is pretty confusing and in my testing wrong. I'm setting custom header for a route response like this:
call.response.headers.append("Test-Header", "Test-Value")
when getting header values the output is
{Test-Header=[Test-Value, Test-Value, Test-Value]}
When adding additional header with the same name:
call.response.headers.append("Test-Header", "Test-Value2")
the output is:
{Test-Header=[Test-Value, Test-Value, Test-Value, Test-Value2, Test-Value, Test-Value, Test-Value, Test-Value2]}
Output when using netty server:
{Test-Header=[Test-Value]}
{Test-Header=[Test-Value, Test-Value2, Test-Value, Test-Value2]}
*Second output is also correct since there are 2 headers with 2 values but with the same key
To reproduce:
Create simple CIO and Netty servers, add headers on the route and get response headers via allValues() function.
Or clone the project https://github.com/Theanko1412/ktor-issue-demo
Additional information:
After looking at git history this function was changed in commit from 2017, I mocked old behavior of this function with allValuesFixed() in my demo project and it is working as expected.
The response is having correct values, problem is only in retrieving it with allValues()
Proposed solution:
Revert function to its previous state.
If someone can provide some more information why was this changed and how was current implementation meant to be used I would appreciate it. Currently don't see any other way to correctly retrieve all headers from a response other than this.
Issue seems to be pretty old but couldn't find any old issues about this, if this is duplicate I would appreciate if you can point me to the solution and reasons why am I getting triple header value.
YAML properties with literal value null cannot be read since 2.3.1
Hello,
since version 2.3.1 (I suspect it came with the change https://youtrack.jetbrains.com/issue/KTOR-5797, but that's just a guess), it is not possible anymore to access a YAML configuration value that is assigned an explicit null value via the propertyOrNull method.
Example:
ktor:
application:
modules:
- com.example.ApplicationKt.module
deployment:
port: 8080
sample:
prop1: null
# pro2: null
package com.example
import io.ktor.server.application.*
import io.ktor.server.response.*
import io.ktor.server.routing.*
fun main(args: Array<String>) {
io.ktor.server.netty.EngineMain.main(args)
}
fun Application.module() {
routing {
get("/") {
val config = call.application.environment.config
call.application.log.info("prop1: ${config.propertyOrNull("ktor.sample.prop1") == null}")
call.application.log.info("prop2: ${config.propertyOrNull("ktor.sample.prop2") == null}")
call.respondText("OK")
}
}
}
With up to version 2.3.0, it runs fine, but starting at 2.3.1, it fails as follows:
ApplicationConfigurationException: Expected primitive or list at path ktor.sample.prop1, but was class net.mamoe.yamlkt.YamlNull
I suspect that's the case because, in the YamlConfig#propertyOrNull implementation the line val value = yaml[parts.last()] ?: return null does not work as intended because the evaluation of the expression yaml[parts.last()] yields an instance of YamlNull instead of null itself. Therefore, the early exit with the evlis operator ?: return null does not fire. But again, that's just a suspicion of mine.
I am not sure if this is entirely intentional or if it indicates a bug. If it does, I would be glad if you could fix it. My temporary workaround is commenting out configuration variables whose values should be null instead of explicitly assigning null because then it behaves as previously.
I have attached the repro example from above as zip.
Test Infrastructure
Resolved connectors job does not complete in TestApplicationEngine
BaseApplicationEngine launches job on application coroutine context which awaits resolvedConnectors to complete. They are completed successfully on start with NettyApplicationEngine but never completes with TestApplicationEngine.
In tests, we'd like to await all the async tasks that were started with code:
withTimeoutOrNull(timeout) { application.coroutineContext.job.children.forEach { scope -> scope.join() } } ?: throw TimeoutException("Timed out waiting ($timeout) for child coroutines to finish")
but the resolvedConnectors job is still awaiting resolved connectors.
As a workaround, the first thing we do (after engine initialization) is
engine.application.coroutineContext.job.children.firstOrNull()?.cancelAndJoin()
but this is hacky a little bit.
Other
KTor 2.3.5 Kotlin 1.9.x upgrade is a breaking change
Our project uses Kotlin 1.7. It's now unable to benefit from this "bugfix release", due to binary incompatibility.
e.g.
lass 'io.ktor.http.HttpHeaders' was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.9.0, expected version is 1.7.1.