Changelog 2.3 version
2.3.13
released 21st November 2024
Other
Add watchosDeviceArm64 target
According to the doc, target `watchosDeviceArm64` is missing
{width=70%}
And the project with this target enabled can't be built
{width=70%}
See https://youtrack.jetbrains.com/issue/KT-53107/Add-arm64-support-for-watchOS-targets-Xcode-14 for reference
CIO: Requests face connection timeouts when executed on the Android main dispatcher
In some cases the engine stops being able to execute any requests, instead failing all of them with a timeout.
I traced this to the semaphores in Endpoint and ConnectionFactory not being released when a request is made from the main thread. Endpoint.releaseConnection() attempts to get the address to be released using InetSocketAddress(host, port), which throws android.os.NetworkOnMainThreadException. This gets caught and doesn't cause a crash, but it does skip the semaphore releases, and so over time all of their permits get used up and once they run out, all subsequent requests will get stuck on these semaphores, never actually getting a permit and eventually failing with a timeout.
To reproduce, create a client using the cio engine, and set the engine's maxConnectionsCount and endpoint.maxConnectionsPerRoute to some smaller number, so you don't have to wait forever. Then make a bunch of requests from the main thread and wait until they start failing.
Now whether or not making requests from the main thread has any merit is perhaps a question to be considered, but this is not acceptable behavior either way - ktor should either throw an exception before even attempting the request, or do it properly. I personally don't see a reason to restrict this, given that the main thread is not actually used for the io (not intentionally anyway).
I include a small patch which fixes the issue for me here for your consideration. I can't currently test this very well with the latest main branch, but this has worked fine in production with ktor 2.3.8 for a few weeks now. The idea is simple: Just store the resolved address in a variable, so we can use the exact same instance later on instead of resolving it again.
diff --git a/ktor-client/ktor-client-cio/jvmAndNix/src/io/ktor/client/engine/cio/Endpoint.kt b/ktor-client/ktor-client-cio/jvmAndNix/src/io/ktor/client/engine/cio/Endpoint.kt
index cd099505e..81c1db0e5 100644
--- a/ktor-client/ktor-client-cio/jvmAndNix/src/io/ktor/client/engine/cio/Endpoint.kt
+++ b/ktor-client/ktor-client-cio/jvmAndNix/src/io/ktor/client/engine/cio/Endpoint.kt
@@ -37,6 +37,8 @@ internal class Endpoint(
private val deliveryPoint: Channel<RequestTask> = Channel()
private val maxEndpointIdleTime: Long = 2 * config.endpoint.connectTimeout
+ private lateinit var address: InetSocketAddress
+
private val timeout = launch(coroutineContext + CoroutineName("Endpoint timeout($host:$port)")) {
try {
while (true) {
@@ -59,6 +61,7 @@ internal class Endpoint(
callContext: CoroutineContext
): HttpResponseData {
lastActivity.value = getTimeMillis()
+ address = InetSocketAddress(host, port)
if (!config.pipelining || request.requiresDedicatedConnection()) {
return makeDedicatedRequest(request, callContext)
@@ -201,8 +204,6 @@ internal class Endpoint(
try {
repeat(connectAttempts) {
- val address = InetSocketAddress(host, port)
-
val connect: suspend CoroutineScope.() -> Socket = {
connectionFactory.connect(address) {
this.socketTimeout = socketTimeout
@@ -284,7 +285,6 @@ internal class Endpoint(
}
private fun releaseConnection() {
- val address = InetSocketAddress(host, port)
connectionFactory.release(address)
connections.decrementAndGet()
}
Replace custom withTimeout implementation using WeakTimeoutQueue with coroutines.withTimeout
Check, if this custom WeakTimeoutQueue is still needed with coroutines 1.6.0 or could be replaced by the existing coroutines.withTimeout function.
Reason:
- remove "outdated" workarounds and use more coroutines apis
- coroutines.withTimeout does not need a GMTClock, but uses CoroutineScheduler
io.ktor.util.TextKt.chomp doesn't work on strings with more than one character
chomp assumes the input string is exactly one character, which is understandable given its usage within Ktor, but since it's public, I would expect it to work with any non-empty separator.
The fix for this is trivial, and I would make a PR for it myself, but Gradle gets consumed by OOM before Ktor build finishes on my machine. :/
"java.lang.IllegalArgumentException: Failed requirement." in SelectorManagerSupport
Some of my Android users are getting the following exception:
Fatal Exception: java.lang.IllegalArgumentException: Failed requirement.
at io.ktor.network.selector.SelectorManagerSupport.select(SelectorManagerSupport.java:34)
at io.ktor.network.sockets.DatagramSocketImpl.receiveSuspend(DatagramSocketImpl.java:77)
at io.ktor.network.sockets.DatagramSocketImpl.receiveImpl(DatagramSocketImpl.java:66)
at io.ktor.network.sockets.DatagramSocketImpl.access$receiveImpl(DatagramSocketImpl.java:16)
at io.ktor.network.sockets.DatagramSocketImpl$receiver$1.invokeSuspend(DatagramSocketImpl.java:40)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(BaseContinuationImpl.java:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.java:106)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.java:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.java:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.java:665)
There are just a few users who get this exception. I am not able to reproduce the issue myself so it would be a bit hard to create a sample project. I am using Ktor UDP sockets which seem to be the cause by looking at the stack trace.
My app uses a manually built version of Ktor based on the following commit: https://github.com/Thomas-Vos/ktor/tree/94fab7c9411414506c27ba3e120af64210535d5e
Any ideas why this could be happening? It would seem to me that this type of exception should never happen.
2.3.12
released 21st June 2024
Other
NoSuchMethodError when using coroutines 1.9.0-RC
To reproduce, run the following code while using the kotlinx-coroutines-core:1.9.0-RC:
embeddedServer(Netty, port = 3333) {}.start()
As a result, the following exception is thrown:
Exception in thread "main" java.lang.NoSuchMethodError: 'void kotlinx.coroutines.internal.LockFreeLinkedListHead.addLast(kotlinx.coroutines.internal.LockFreeLinkedListNode)'
at io.ktor.events.Events.subscribe(Events.kt:24)
at io.ktor.server.engine.BaseApplicationEngine.<init>(BaseApplicationEngine.kt:50)
at io.ktor.server.engine.BaseApplicationEngine.<init>(BaseApplicationEngine.kt:31)
at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:33)
at io.ktor.server.netty.Netty.create(Embedded.kt:18)
at io.ktor.server.netty.Netty.create(Embedded.kt:13)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:111)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:100)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:65)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:40)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer$default(EmbeddedServer.kt:32)
Docs: Update versions in codeSnippets
Most importantly, the kotlin coroutines and serialization versions are wrong and need to be set to the latest available versions.
Gradle needs to be updated to v8.
OpenTelemetry error spans for successful requests
The new version of OpenTelemetry produces error spans for successful responses with exception. I suppose that it happens for all http requests
kotlinx.coroutines.JobCancellationException: JobImpl has completed normally; job=JobImpl{Completed}@2da13f92
at kotlinx.coroutines.JobSupport.getCancellationException(JobSupport.kt:422)
at io.opentelemetry.instrumentation.ktor.v2_0.client.KtorClientTracing$Companion$installSpanEnd$1$1.invokeSuspend(KtorClientTracing.kt:101)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith$$$capture(ContinuationImpl.kt:33)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
Server: Content-Type header for static js, css and svg resources misses charset
For example:
Static .js files are served with Content-Type: text/javascript instead of Content-Type: text/javascript; charset=utf-8.
Same issue with .css and .svg.
Methods in FakeTaskRepository in first iteration shouldn't have the suspend keyword
In this tutorial: https://ktor.io/docs/server-integrate-database.html#add-starter-code
The methods ofFakeTaskRepository in the first iteration should not have the suspend keyword.
The code should be as follows:
class FakeTaskRepository : TaskRepository {
private val tasks = mutableListOf(
Task("cleaning", "Clean the house", Priority.Low),
Task("gardening", "Mow the lawn", Priority.Medium),
Task("shopping", "Buy the groceries", Priority.High),
Task("painting", "Paint the fence", Priority.Medium)
)
override fun allTasks(): List<Task> = tasks
override fun tasksByPriority(priority: Priority) = tasks.filter {
it.priority == priority
}
override fun taskByName(name: String) = tasks.find {
it.name.equals(name, ignoreCase = true)
}
override fun addTask(task: Task) {
if (taskByName(task.name) != null) {
throw IllegalStateException("Cannot duplicate task names!")
}
tasks.add(task)
}
override fun removeTask(name: String): Boolean {
return tasks.removeIf { it.name == name }
}
}
OpenTelemetry: Incorrect end time of span when receiving response body
Currently, OpenTelemetry client tracing is based on HttpReceivePipeline.After. This phase is called after the receiving of response headers. As a result, we measure incorrect time of request and this difference can be extremely big in the case of log-running calls, for example, SSE.
Documentation Restructuring: Add new content
Implement the changes necessary as described in Documentation Restructuring from June 2023.
Existing tutorials are to be replaced with the following:
- Getting Started Guide
- Routing and Requests
- Content Negotiation and REST
- Building Web Applications
- Working with WebSockets
- Database with Exposed
- Security with OAuth
- Cloud Deployment and Configuration
Add a new tutorial 'Full Stack development with Kotlin Multiplatform'
The content is available here
Add a new tutorial 'First steps with Kotlin RPC'
Transfer the content from the tutorial 'Fist steps with Kotlin RPC` into the Ktor Docs.
To be placed under a new topic section called 'Integrations'.
Improve the docs by showcasing broadcasting messages to WebSocket clients via SharedFlow
Hello Ktor team,
I've been working with WebSockets in Ktor and noticed an opportunity for improvement in how broadcast messages are handled. Currently, maintaining a list of WebSocket connections for broadcasting can be cumbersome. I'd like to propose using SharedFlow as an alternative approach.
https://ktor.io/docs/server-websockets.html#handle-multiple-session
Current approach:
- Maintain a list of WebSocket connections
- Iterate through the list to send broadcast messages
Proposed improvement:
- Use a
SharedFlowto manage broadcast messages - Each WebSocket connection collects from the
SharedFlow
Benefits:
- Simplified code structure
- Better handling of concurrency
- Improved scalability for multiple connections
- Easier to manage connection lifecycles
I've implemented a proof of concept using SharedFlow that works well. Here's a simplified version of the code:
fun Route.routingWS() {
val messageFlow = MutableSharedFlow<Message>()
val sharedFlow = messageFlow.asSharedFlow()
webSocket("/ws") {
send("You are connected to WebSocket!")
val job = launch {
sharedFlow.collect { message ->
send("Broadcast: ${message.message}")
}
}
try {
incoming.consumeEach { frame ->
if (frame is Frame.Text) {
val receivedText = frame.readText()
val message = Message(receivedText)
messageFlow.emit(message)
send("You said: $receivedText")
}
}
} finally {
job.cancel()
}
}
}
I believe this approach could be beneficial for Ktor users working with WebSockets.
Question : Would you consider incorporating this pattern into the Ktor documentation or examples?
Example project: https://github.com/mbakgun/dcbln24-mbakgun/
I appreciate your consideration.
Update Gradle version in Ktor-samples
Currently, you get an error if you are trying to import samples projects
{width=70%}
Updating the Gradle version will fix the problem, but I think we can solve this problem in general because all users who want to update the Ktor Gradle plugin will encounter it
@leonid.stashevsky
Embedded Linux device without iso-8859-1 and UTF-16 cannot use ktor-network
I'm trying to use ktor-network to speak UTF-8 to a unix domain socket. The Charsets object on linux native eagerly initializes three charsets using iconv: UTF-8, ISO-8859-1, and UTF-16 (either BE or LE). My embedded Linux device only supports UTF-8, and both ISO-8859-1 and UTF-16BE fail preventing any use of ktor (tested manually using iconv_open).
Since ISO 8859-1 is only used by HTTP headers and I have no idea what UTF-16 is used for, could those be made lazy? Or perhaps all of them should be lazily initialized? Or maybe only UTF-8 should be provided at this layer and the HTTP module should cache its own ISO 8859-1 instance and whatever uses UTF-16 could cache its own instance?
Caused by: kotlin.IllegalArgumentException: Failed to open iconv for charset ISO-8859-1 with error code 22
at 0 example.kexe 0xea833b kfun:kotlin.Throwable#<init>(kotlin.String?){} + 91
at 1 example.kexe 0xea2ec3 kfun:kotlin.Exception#<init>(kotlin.String?){} + 83
at 2 example.kexe 0xea3093 kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 83
at 3 example.kexe 0xea3263 kfun:kotlin.IllegalArgumentException#<init>(kotlin.String?){} + 83
at 4 example.kexe 0x14df68f kfun:io.ktor.utils.io.charsets#checkErrors(kotlinx.cinterop.CPointer<out|kotlinx.cinterop.CPointed>?;kotlin.String){} + 639
at 5 example.kexe 0x14df0c3 kfun:io.ktor.utils.io.charsets.CharsetIconv.<init>#internal + 515
at 6 example.kexe 0x14ded53 kfun:io.ktor.utils.io.charsets.Charsets#<init>(){} + 275
at 7 example.kexe 0x14dec03 kfun:io.ktor.utils.io.charsets.Charsets.$init_global#internal + 147
at 8 example.kexe 0x15dc723 CallInitGlobalPossiblyLock + 487
at 9 example.kexe 0x14dee87 kfun:io.ktor.utils.io.charsets.Charsets#<get-$instance>#static(){}io.ktor.utils.io.charsets.Charsets + 71
Note: I'm actually using 3.0.0-beta-1 but I'm not allowed to change the affected versions for some reason.
Update dependency on swagger
Now the ktor swagger plugin does not support Open Api 3.1.*. Swagger supports 3.1.
We could update the dependency on swagger in the plugin io.ktor:ktor-server-swagger
2.3.11
released 9th May 2024
Other
Sessions: documentation snippet lacks import statement
UPD: actually, I wrote initialization code in the wrong lambda. ktor IDEA plugin generates this main function:
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0", module = Application::module)
.start(wait = true)
}
while the documentation snippet here defines configuration inside the lambda parameter:
fun main() {
embeddedServer(Netty, port = 8080) {
install(Sessions)
// ...
}.start(wait = true)
}
But the embeddedServer function has another lambda parameter and I tried to call install(Sessions) there, which resulted in error:
fun main() {
db.apply {}
embeddedServer(Netty, port = 8080, host = "0.0.0.0", module = Application::module) {
// compilation error here:
install(Sessions)
}
.start(wait = true)
}
Looks like the documentation should be aligned with the template or vice versa.
Ktor client with logback-classic on android produces compilation error on build
Hi,
We've been using Ktor client in our android project with a very old version of logback which broke after it was updated to 1.4.6 + Android Gradle Plugin 8.0.0.
The error was the same described in this SO post (2 files found with path 'META-INF/INDEX.LIST' from inputs):
https://stackoverflow.com/questions/75704844/how-to-set-up-ktor-logging-in-android
* What went wrong:
Execution failed for task ':app:mergeDebugJavaResource'.
> A failure occurred while executing com.android.build.gradle.internal.tasks.MergeJavaResWorkAction
> 2 files found with path 'META-INF/INDEX.LIST' from inputs:
- D:\Software\.gradle\caches\modules-2\files-2.1\ch.qos.logback\logback-classic\1.4.5\28e7dc0b208d6c3f15beefd73976e064b4ecfa9b\logback-classic-1.4.5.jar
- D:\Software\.gradle\caches\modules-2\files-2.1\ch.qos.logback\logback-core\1.4.5\e9bb2ea70f84401314da4300343b0a246c8954da\logback-core-1.4.5.jar
Adding a packagingOptions block may help, please refer to
https://developer.android.com/reference/tools/gradle-api/7.4/com/android/build/api/dsl/ResourcesPackagingOptions
for more information
In this SO post Aleksei suggested to use org.slf4j:slf4j-android:1.7.36 SLF4J Android Binding library instead of the logback-classic which worked for us and the guy in the SO post too.
If this is the official recommendation for android could you please update the relevant docs with this library so others can use it straight away.
Thank you!
Update the procedure and screenshots in "Creating a client application" tutorial
https://ktor.io/docs/client-create-new-application.html
IDEA doesn't have the Kotlin Multiplatform template anymore. The project should be created from a Kotlin template instead.
Broken link in the "Integrate a database" topic
On the "Integrate a database with Kotlin, Ktor, and Exposed" page the link tutorial-server-database-integration to the sample leads to the 404 page.
Replace the tutorials under 'Creating a website' with new content
The material for the two pages within 'Creating a Website' has been combined into Building Web Applications. If you want to preserve existing titles and URL's you could still call this Creating An Interactive Website. NB no one would use a framework like Ktor to create a static website as described here. It's the wrong tool for the job,
Replace the 'Creating a WebSocket chat' tutorial with new content
The Creating a Website Chat page is replaced by Working With WebSockets. This new tutorial continues and extends the Task Manager case study from the previous tutorials. So it should be much easier for readers understand.
Add a new tutorial 'Database integration with Exposed'
The content is available here
Replace Creating HTTP API's tutorial with new content
The material from the Creating HTTP API's page is to be split into Understanding Routing and Requests and Content Negotiation and REST.
You could still use the old name and URL for the first of these tutorials. It's close enough.
Test client ignores socket timeout
I would expect the following test to pass, but it looks like socket timeout is ignored by the test client.
class ApplicationTest {
@Test
fun `test socketTimeout`() = testApplication {
routing {
get("/") {
call.respondOutputStream {
write("Hello World".toByteArray())
delay(10000)
}
}
}
val clientWithTimeout = client.config {
install(HttpTimeout) {
socketTimeoutMillis = 1_000
}
}
assertFails {
println(clientWithTimeout.get("/").readBytes().decodeToString())
}
}
}