Changelog 2.3 version
2.3.13
released 21st November 2024
Client
CIO: Requests face connection timeouts when executed on the Android main dispatcher
In some cases the engine stops being able to execute any requests, instead failing all of them with a timeout.
I traced this to the semaphores in Endpoint and ConnectionFactory not being released when a request is made from the main thread. Endpoint.releaseConnection() attempts to get the address to be released using InetSocketAddress(host, port), which throws android.os.NetworkOnMainThreadException. This gets caught and doesn't cause a crash, but it does skip the semaphore releases, and so over time all of their permits get used up and once they run out, all subsequent requests will get stuck on these semaphores, never actually getting a permit and eventually failing with a timeout.
To reproduce, create a client using the cio engine, and set the engine's maxConnectionsCount and endpoint.maxConnectionsPerRoute to some smaller number, so you don't have to wait forever. Then make a bunch of requests from the main thread and wait until they start failing.
Now whether or not making requests from the main thread has any merit is perhaps a question to be considered, but this is not acceptable behavior either way - ktor should either throw an exception before even attempting the request, or do it properly. I personally don't see a reason to restrict this, given that the main thread is not actually used for the io (not intentionally anyway).
I include a small patch which fixes the issue for me here for your consideration. I can't currently test this very well with the latest main branch, but this has worked fine in production with ktor 2.3.8 for a few weeks now. The idea is simple: Just store the resolved address in a variable, so we can use the exact same instance later on instead of resolving it again.
diff --git a/ktor-client/ktor-client-cio/jvmAndNix/src/io/ktor/client/engine/cio/Endpoint.kt b/ktor-client/ktor-client-cio/jvmAndNix/src/io/ktor/client/engine/cio/Endpoint.kt
index cd099505e..81c1db0e5 100644
--- a/ktor-client/ktor-client-cio/jvmAndNix/src/io/ktor/client/engine/cio/Endpoint.kt
+++ b/ktor-client/ktor-client-cio/jvmAndNix/src/io/ktor/client/engine/cio/Endpoint.kt
@@ -37,6 +37,8 @@ internal class Endpoint(
private val deliveryPoint: Channel<RequestTask> = Channel()
private val maxEndpointIdleTime: Long = 2 * config.endpoint.connectTimeout
+ private lateinit var address: InetSocketAddress
+
private val timeout = launch(coroutineContext + CoroutineName("Endpoint timeout($host:$port)")) {
try {
while (true) {
@@ -59,6 +61,7 @@ internal class Endpoint(
callContext: CoroutineContext
): HttpResponseData {
lastActivity.value = getTimeMillis()
+ address = InetSocketAddress(host, port)
if (!config.pipelining || request.requiresDedicatedConnection()) {
return makeDedicatedRequest(request, callContext)
@@ -201,8 +204,6 @@ internal class Endpoint(
try {
repeat(connectAttempts) {
- val address = InetSocketAddress(host, port)
-
val connect: suspend CoroutineScope.() -> Socket = {
connectionFactory.connect(address) {
this.socketTimeout = socketTimeout
@@ -284,7 +285,6 @@ internal class Endpoint(
}
private fun releaseConnection() {
- val address = InetSocketAddress(host, port)
connectionFactory.release(address)
connections.decrementAndGet()
}
Core
"java.lang.IllegalArgumentException: Failed requirement." in SelectorManagerSupport
Some of my Android users are getting the following exception:
Fatal Exception: java.lang.IllegalArgumentException: Failed requirement.
at io.ktor.network.selector.SelectorManagerSupport.select(SelectorManagerSupport.java:34)
at io.ktor.network.sockets.DatagramSocketImpl.receiveSuspend(DatagramSocketImpl.java:77)
at io.ktor.network.sockets.DatagramSocketImpl.receiveImpl(DatagramSocketImpl.java:66)
at io.ktor.network.sockets.DatagramSocketImpl.access$receiveImpl(DatagramSocketImpl.java:16)
at io.ktor.network.sockets.DatagramSocketImpl$receiver$1.invokeSuspend(DatagramSocketImpl.java:40)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(BaseContinuationImpl.java:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.java:106)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.java:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.java:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.java:665)
There are just a few users who get this exception. I am not able to reproduce the issue myself so it would be a bit hard to create a sample project. I am using Ktor UDP sockets which seem to be the cause by looking at the stack trace.
My app uses a manually built version of Ktor based on the following commit: https://github.com/Thomas-Vos/ktor/tree/94fab7c9411414506c27ba3e120af64210535d5e
Any ideas why this could be happening? It would seem to me that this type of exception should never happen.
Server
Replace custom withTimeout implementation using WeakTimeoutQueue with coroutines.withTimeout
Check, if this custom WeakTimeoutQueue is still needed with coroutines 1.6.0 or could be replaced by the existing coroutines.withTimeout function.
Reason:
- remove "outdated" workarounds and use more coroutines apis
- coroutines.withTimeout does not need a GMTClock, but uses CoroutineScheduler
Other
Add watchosDeviceArm64 target
According to the doc, target `watchosDeviceArm64` is missing
{width=70%}
And the project with this target enabled can't be built
{width=70%}
See https://youtrack.jetbrains.com/issue/KT-53107/Add-arm64-support-for-watchOS-targets-Xcode-14 for reference
io.ktor.util.TextKt.chomp doesn't work on strings with more than one character
chomp assumes the input string is exactly one character, which is understandable given its usage within Ktor, but since it's public, I would expect it to work with any non-empty separator.
The fix for this is trivial, and I would make a PR for it myself, but Gradle gets consumed by OOM before Ktor build finishes on my machine. :/
2.3.12
released 21st June 2024
Client
OpenTelemetry error spans for successful requests
The new version of OpenTelemetry produces error spans for successful responses with exception. I suppose that it happens for all http requests
kotlinx.coroutines.JobCancellationException: JobImpl has completed normally; job=JobImpl{Completed}@2da13f92
at kotlinx.coroutines.JobSupport.getCancellationException(JobSupport.kt:422)
at io.opentelemetry.instrumentation.ktor.v2_0.client.KtorClientTracing$Companion$installSpanEnd$1$1.invokeSuspend(KtorClientTracing.kt:101)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith$$$capture(ContinuationImpl.kt:33)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
OpenTelemetry: Incorrect end time of span when receiving response body
Currently, OpenTelemetry client tracing is based on HttpReceivePipeline.After. This phase is called after the receiving of response headers. As a result, we measure incorrect time of request and this difference can be extremely big in the case of log-running calls, for example, SSE.
Core
Embedded Linux device without iso-8859-1 and UTF-16 cannot use ktor-network
I'm trying to use ktor-network to speak UTF-8 to a unix domain socket. The Charsets object on linux native eagerly initializes three charsets using iconv: UTF-8, ISO-8859-1, and UTF-16 (either BE or LE). My embedded Linux device only supports UTF-8, and both ISO-8859-1 and UTF-16BE fail preventing any use of ktor (tested manually using iconv_open).
Since ISO 8859-1 is only used by HTTP headers and I have no idea what UTF-16 is used for, could those be made lazy? Or perhaps all of them should be lazily initialized? Or maybe only UTF-8 should be provided at this layer and the HTTP module should cache its own ISO 8859-1 instance and whatever uses UTF-16 could cache its own instance?
Caused by: kotlin.IllegalArgumentException: Failed to open iconv for charset ISO-8859-1 with error code 22
at 0 example.kexe 0xea833b kfun:kotlin.Throwable#<init>(kotlin.String?){} + 91
at 1 example.kexe 0xea2ec3 kfun:kotlin.Exception#<init>(kotlin.String?){} + 83
at 2 example.kexe 0xea3093 kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 83
at 3 example.kexe 0xea3263 kfun:kotlin.IllegalArgumentException#<init>(kotlin.String?){} + 83
at 4 example.kexe 0x14df68f kfun:io.ktor.utils.io.charsets#checkErrors(kotlinx.cinterop.CPointer<out|kotlinx.cinterop.CPointed>?;kotlin.String){} + 639
at 5 example.kexe 0x14df0c3 kfun:io.ktor.utils.io.charsets.CharsetIconv.<init>#internal + 515
at 6 example.kexe 0x14ded53 kfun:io.ktor.utils.io.charsets.Charsets#<init>(){} + 275
at 7 example.kexe 0x14dec03 kfun:io.ktor.utils.io.charsets.Charsets.$init_global#internal + 147
at 8 example.kexe 0x15dc723 CallInitGlobalPossiblyLock + 487
at 9 example.kexe 0x14dee87 kfun:io.ktor.utils.io.charsets.Charsets#<get-$instance>#static(){}io.ktor.utils.io.charsets.Charsets + 71
Note: I'm actually using 3.0.0-beta-1 but I'm not allowed to change the affected versions for some reason.
Docs
Docs: Update versions in codeSnippets
Most importantly, the kotlin coroutines and serialization versions are wrong and need to be set to the latest available versions.
Gradle needs to be updated to v8.
Methods in FakeTaskRepository in first iteration shouldn't have the suspend keyword
In this tutorial: https://ktor.io/docs/server-integrate-database.html#add-starter-code
The methods ofFakeTaskRepository in the first iteration should not have the suspend keyword.
The code should be as follows:
class FakeTaskRepository : TaskRepository {
private val tasks = mutableListOf(
Task("cleaning", "Clean the house", Priority.Low),
Task("gardening", "Mow the lawn", Priority.Medium),
Task("shopping", "Buy the groceries", Priority.High),
Task("painting", "Paint the fence", Priority.Medium)
)
override fun allTasks(): List<Task> = tasks
override fun tasksByPriority(priority: Priority) = tasks.filter {
it.priority == priority
}
override fun taskByName(name: String) = tasks.find {
it.name.equals(name, ignoreCase = true)
}
override fun addTask(task: Task) {
if (taskByName(task.name) != null) {
throw IllegalStateException("Cannot duplicate task names!")
}
tasks.add(task)
}
override fun removeTask(name: String): Boolean {
return tasks.removeIf { it.name == name }
}
}
Documentation Restructuring: Add new content
Implement the changes necessary as described in Documentation Restructuring from June 2023.
Existing tutorials are to be replaced with the following:
- Getting Started Guide
- Routing and Requests
- Content Negotiation and REST
- Building Web Applications
- Working with WebSockets
- Database with Exposed
- Security with OAuth
- Cloud Deployment and Configuration
Add a new tutorial 'Full Stack development with Kotlin Multiplatform'
The content is available here
Add a new tutorial 'First steps with Kotlin RPC'
Transfer the content from the tutorial 'Fist steps with Kotlin RPC` into the Ktor Docs.
To be placed under a new topic section called 'Integrations'.
Improve the docs by showcasing broadcasting messages to WebSocket clients via SharedFlow
Hello Ktor team,
I've been working with WebSockets in Ktor and noticed an opportunity for improvement in how broadcast messages are handled. Currently, maintaining a list of WebSocket connections for broadcasting can be cumbersome. I'd like to propose using SharedFlow as an alternative approach.
https://ktor.io/docs/server-websockets.html#handle-multiple-session
Current approach:
- Maintain a list of WebSocket connections
- Iterate through the list to send broadcast messages
Proposed improvement:
- Use a
SharedFlowto manage broadcast messages - Each WebSocket connection collects from the
SharedFlow
Benefits:
- Simplified code structure
- Better handling of concurrency
- Improved scalability for multiple connections
- Easier to manage connection lifecycles
I've implemented a proof of concept using SharedFlow that works well. Here's a simplified version of the code:
fun Route.routingWS() {
val messageFlow = MutableSharedFlow<Message>()
val sharedFlow = messageFlow.asSharedFlow()
webSocket("/ws") {
send("You are connected to WebSocket!")
val job = launch {
sharedFlow.collect { message ->
send("Broadcast: ${message.message}")
}
}
try {
incoming.consumeEach { frame ->
if (frame is Frame.Text) {
val receivedText = frame.readText()
val message = Message(receivedText)
messageFlow.emit(message)
send("You said: $receivedText")
}
}
} finally {
job.cancel()
}
}
}
I believe this approach could be beneficial for Ktor users working with WebSockets.
Question : Would you consider incorporating this pattern into the Ktor documentation or examples?
Example project: https://github.com/mbakgun/dcbln24-mbakgun/
I appreciate your consideration.
Samples
Update Gradle version in Ktor-samples
Currently, you get an error if you are trying to import samples projects
{width=70%}
Updating the Gradle version will fix the problem, but I think we can solve this problem in general because all users who want to update the Ktor Gradle plugin will encounter it
@leonid.stashevsky
Server
NoSuchMethodError when using coroutines 1.9.0-RC
To reproduce, run the following code while using the kotlinx-coroutines-core:1.9.0-RC:
embeddedServer(Netty, port = 3333) {}.start()
As a result, the following exception is thrown:
Exception in thread "main" java.lang.NoSuchMethodError: 'void kotlinx.coroutines.internal.LockFreeLinkedListHead.addLast(kotlinx.coroutines.internal.LockFreeLinkedListNode)'
at io.ktor.events.Events.subscribe(Events.kt:24)
at io.ktor.server.engine.BaseApplicationEngine.<init>(BaseApplicationEngine.kt:50)
at io.ktor.server.engine.BaseApplicationEngine.<init>(BaseApplicationEngine.kt:31)
at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:33)
at io.ktor.server.netty.Netty.create(Embedded.kt:18)
at io.ktor.server.netty.Netty.create(Embedded.kt:13)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:111)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:100)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:65)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:40)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer$default(EmbeddedServer.kt:32)
Server: Content-Type header for static js, css and svg resources misses charset
For example:
Static .js files are served with Content-Type: text/javascript instead of Content-Type: text/javascript; charset=utf-8.
Same issue with .css and .svg.
Update dependency on swagger
Now the ktor swagger plugin does not support Open Api 3.1.*. Swagger supports 3.1.
We could update the dependency on swagger in the plugin io.ktor:ktor-server-swagger
2.3.11
released 9th May 2024
Docs
Sessions: documentation snippet lacks import statement
UPD: actually, I wrote initialization code in the wrong lambda. ktor IDEA plugin generates this main function:
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0", module = Application::module)
.start(wait = true)
}
while the documentation snippet here defines configuration inside the lambda parameter:
fun main() {
embeddedServer(Netty, port = 8080) {
install(Sessions)
// ...
}.start(wait = true)
}
But the embeddedServer function has another lambda parameter and I tried to call install(Sessions) there, which resulted in error:
fun main() {
db.apply {}
embeddedServer(Netty, port = 8080, host = "0.0.0.0", module = Application::module) {
// compilation error here:
install(Sessions)
}
.start(wait = true)
}
Looks like the documentation should be aligned with the template or vice versa.
Ktor client with logback-classic on android produces compilation error on build
Hi,
We've been using Ktor client in our android project with a very old version of logback which broke after it was updated to 1.4.6 + Android Gradle Plugin 8.0.0.
The error was the same described in this SO post (2 files found with path 'META-INF/INDEX.LIST' from inputs):
https://stackoverflow.com/questions/75704844/how-to-set-up-ktor-logging-in-android
* What went wrong:
Execution failed for task ':app:mergeDebugJavaResource'.
> A failure occurred while executing com.android.build.gradle.internal.tasks.MergeJavaResWorkAction
> 2 files found with path 'META-INF/INDEX.LIST' from inputs:
- D:\Software\.gradle\caches\modules-2\files-2.1\ch.qos.logback\logback-classic\1.4.5\28e7dc0b208d6c3f15beefd73976e064b4ecfa9b\logback-classic-1.4.5.jar
- D:\Software\.gradle\caches\modules-2\files-2.1\ch.qos.logback\logback-core\1.4.5\e9bb2ea70f84401314da4300343b0a246c8954da\logback-core-1.4.5.jar
Adding a packagingOptions block may help, please refer to
https://developer.android.com/reference/tools/gradle-api/7.4/com/android/build/api/dsl/ResourcesPackagingOptions
for more information
In this SO post Aleksei suggested to use org.slf4j:slf4j-android:1.7.36 SLF4J Android Binding library instead of the logback-classic which worked for us and the guy in the SO post too.
If this is the official recommendation for android could you please update the relevant docs with this library so others can use it straight away.
Thank you!
Update the procedure and screenshots in "Creating a client application" tutorial
https://ktor.io/docs/client-create-new-application.html
IDEA doesn't have the Kotlin Multiplatform template anymore. The project should be created from a Kotlin template instead.
Broken link in the "Integrate a database" topic
On the "Integrate a database with Kotlin, Ktor, and Exposed" page the link tutorial-server-database-integration to the sample leads to the 404 page.
Replace the tutorials under 'Creating a website' with new content
The material for the two pages within 'Creating a Website' has been combined into Building Web Applications. If you want to preserve existing titles and URL's you could still call this Creating An Interactive Website. NB no one would use a framework like Ktor to create a static website as described here. It's the wrong tool for the job,
Replace the 'Creating a WebSocket chat' tutorial with new content
The Creating a Website Chat page is replaced by Working With WebSockets. This new tutorial continues and extends the Task Manager case study from the previous tutorials. So it should be much easier for readers understand.
Add a new tutorial 'Database integration with Exposed'
The content is available here
Replace Creating HTTP API's tutorial with new content
The material from the Creating HTTP API's page is to be split into Understanding Routing and Requests and Content Negotiation and REST.
You could still use the old name and URL for the first of these tutorials. It's close enough.
Test Infrastructure
Test client ignores socket timeout
I would expect the following test to pass, but it looks like socket timeout is ignored by the test client.
class ApplicationTest {
@Test
fun `test socketTimeout`() = testApplication {
routing {
get("/") {
call.respondOutputStream {
write("Hello World".toByteArray())
delay(10000)
}
}
}
val clientWithTimeout = client.config {
install(HttpTimeout) {
socketTimeoutMillis = 1_000
}
}
assertFails {
println(clientWithTimeout.get("/").readBytes().decodeToString())
}
}
}
2.3.10
released 8th April 2024
Client
NodeJS WebSocket client sometimes drops a frame received immediately after handshake
When the server immediately sends a frame back to the client upon handshake, the client sometimes misses it.
Possible root cause
I believe this is due to the fact that the onMessage web socket listener is not registered immediately after the creation of the web socket instance (which performs the handshake). In JS, since things are single-threaded, it's OK to do the following:
val ws = WebSocket(url)
ws.onmessage = { ... }
But in the case of Ktor implementation, there is an awaitConnection() between the handshake and the message listener registration:
https://github.com/ktorio/ktor/blob/2.3.9/ktor-client/ktor-client-core/js/src/io/ktor/client/engine/js/JsClientEngine.kt#L91-L100
val socket: WebSocket = createWebSocket(urlString, request.headers) // performs the handshake
try {
socket.awaitConnection()
} catch (cause: Throwable) {
callContext.cancel(CancellationException("Failed to connect to $urlString", cause))
throw cause
}
val session = JsWebSocketSession(callContext, socket) // registers socket.onmessage in init{} block
I think the presence of this await here allows a frame to arrive before the message event listener is registered, thus dropping the frame.
Reproducer
First, we need to setup a test server that sends a frame upon handshake. For instance, this simple test server uses org.java-websocket:Java-WebSocket:1.5.6:
import org.java_websocket.*
import org.java_websocket.handshake.*
import org.java_websocket.server.*
import java.net.*
import java.nio.*
fun main() {
val port = 12345
val server = EchoWebSocketServer(port)
println("Starting test WS server...")
server.start()
println("Server listening at ws://localhost:$port")
}
internal class EchoWebSocketServer(port: Int = 0) : WebSocketServer(InetSocketAddress(port)) {
override fun onOpen(conn: WebSocket, handshake: ClientHandshake) {
println("Received connection from ${conn.remoteSocketAddress}${conn.resourceDescriptor}")
conn.send("Hello client!")
println("Said hi to the client")
}
override fun onStart() {}
override fun onMessage(conn: WebSocket, message: String?) {}
override fun onMessage(conn: WebSocket, message: ByteBuffer) {}
override fun onError(conn: WebSocket?, ex: Exception?) {}
override fun onClose(conn: WebSocket, code: Int, reason: String?, remote: Boolean) {}
}
Then we can run this test that demonstrates the problem:
import io.ktor.client.*
import io.ktor.client.plugins.websocket.*
import io.ktor.websocket.*
import kotlinx.coroutines.*
import kotlinx.coroutines.test.*
import kotlin.test.*
import kotlin.time.Duration.Companion.minutes
import kotlin.time.Duration.Companion.seconds
private const val port = 12345 // TODO replace with the acutal port used by the test server
private const val nAttempts = 10 // usually the problem occurs after 2-3 iterations
private val frameTimeout = 2.seconds // we can increase this to confirm it's not a matter of time
class SomeJsTest {
@Test
fun testHandshakeResponse() = runTest(timeout = 1.minutes) {
withContext(Dispatchers.Default) { // to avoid delay skipping from runTest (we're testing a real system)
var nMissed = 0
repeat(nAttempts) { attempt ->
val httpClient = HttpClient { install(WebSockets) }
val session = httpClient.webSocketSession("ws://localhost:$port")
try {
println("Expecting immediate hello from the server...")
val helloFrame = withTimeoutOrNull(frameTimeout) {
session.incoming.receive()
}
if (helloFrame == null) {
println("$attempt: MISSED! (Hello frame not received $frameTimeout after handshake)")
nMissed++
} else {
assertIs<Frame.Text>(helloFrame, "should receive a Text frame as echo")
assertEquals("Hello client!", helloFrame.readText(), "should receive echo")
println("$attempt: Got our hello frame this time!")
}
} finally {
session.close()
}
}
if (nMissed > 0) fail("$nMissed/$nAttempts frames were missed")
}
}
}
The output should look like this:
Expecting immediate hello from the server...
0: Got our hello frame this time!
Expecting immediate hello from the server...
1: MISSED! (Hello frame not received 2s after handshake)
Expecting immediate hello from the server...
2: MISSED! (Hello frame not received 2s after handshake)
Expecting immediate hello from the server...
3: Got our hello frame this time!
Expecting immediate hello from the server...
4: Got our hello frame this time!
Expecting immediate hello from the server...
5: Got our hello frame this time!
Expecting immediate hello from the server...
6: Got our hello frame this time!
Expecting immediate hello from the server...
7: MISSED! (Hello frame not received 2s after handshake)
Expecting immediate hello from the server...
8: MISSED! (Hello frame not received 2s after handshake)
Expecting immediate hello from the server...
9: MISSED! (Hello frame not received 2s after handshake)
AssertionError: 5/10 frames were missed
JS browser: "Maximum call stack size exceeded" on HTTP request when targeting es2015
Setting target to es2015 causes JS client to fail in Chrome
Reproducer:
- Create a minimal project with the code below
- Run tests, observe that the tests succeed
- Run
jsBrowserDevelopmentWebpack - Create this html file
<body style="background: black; color: white">
<p>hello world</p>
<script src="/path/to/ktor-bug-project/build/kotlin-webpack/js/developmentExecutable/ktor-bug-project.js"></script>
</body>
- Open this HTML file in Google Chrome and observe the error
build.gradle.kts
plugins {
kotlin("multiplatform") version "2.0.0-Beta5"
}
repositories.mavenCentral()
kotlin {
js(IR) {
@Suppress("OPT_IN_USAGE")
compilerOptions.target.set("es2015")
browser{
binaries.executable()
}
}
sourceSets.jsMain.dependencies {
implementation("io.ktor:ktor-client-js:2.3.9")
}
}
dependencies {
commonTestImplementation(kotlin("test"))
}
main.kt
import io.ktor.client.HttpClient
import io.ktor.client.request.get
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Job
import kotlinx.coroutines.launch
fun main() {
CoroutineScope(Job()).launch {
HttpClient().get("https://www.example.com")
}
}
test.kt
import kotlin.test.Test
class Tests {
@Test
fun runMain() {
main()
}
}
Android: no logs are present in Logcat with `Logger.ANDROID`
Perhaps it's just an issue of documentation, but when I enable logging with HttpClient(Android) no logs is present unless I provide custom logger implementation.
In more details:
When I create HttpClient with Android Engine using either Logger.DEFAULT or Logger.ANDROID no logs is present in the Logcat .
HttpClient(Android) {
if (SdkConfig.isLoggingEnabled) {
install(Logging) {
logger = Logger.DEFAULT
level = LogLevel.ALL
}
}
}
I am adding logging dependency as io.ktor:ktor-client-logging-jvm:1.4.1. As far as I see there is no specific "android" artefact available.
As far as I see both Logger.DEFAULT or Logger.ANDROID uses slf4j for logging, which does nothing on Android by default.
Ideally when Logger.DEFAULT logger is used in Android it should log to Android system log.
And obviously Logger.ANDROID is expected to do this without any further configuration.
P.S. I have not tried yet logging on JVM, but as for me, it make sense to add the mentioning about required slf4j implementation (e.g. logback) in the dependencies there as well.
CIO: "getSubjectAlternativeNames(...) must not be null" when IP-addresses are verified and no SAN in the certificate
My host have only ip and self-signed certificate. My version ktor 2.3.8
My client setup:
client = HttpClient(CIO) {
install(WebSockets){
pingInterval = 5_000
}
engine {
https {
trustManager = object : X509TrustManager {
override fun checkClientTrusted(chain: Array<out X509Certificate>?, authType: String?) { }
override fun checkServerTrusted(chain: Array<out X509Certificate>?, authType: String?) { }
override fun getAcceptedIssuers(): Array<X509Certificate> {
return emptyArray()
}
}
}
}
install(Logging) {
logger = object : Logger {
override fun log(message: String) {
Timber.tag("Socket Log").e(message)
}
}
level = LogLevel.ALL
}
}
But after try connect i have error: certificate.subjectAlternativeNames must not be null
java.lang.NullPointerException: certificate.subjectAlternativeNames must not be null
at io.ktor.network.tls.HostnameUtilsKt.verifyIpInCertificate(HostnameUtils.kt:30)
at io.ktor.network.tls.HostnameUtilsKt.verifyHostnameInCertificate(HostnameUtils.kt:15)
at io.ktor.network.tls.TLSClientHandshake.handleCertificatesAndKeys(TLSClientHandshake.kt:245)
at io.ktor.network.tls.TLSClientHandshake.negotiate(TLSClientHandshake.kt:166)
at io.ktor.network.tls.TLSClientHandshake$negotiate$1.invokeSuspend(Unknown Source:18)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@abda18f, Dispatchers.IO]
Docs
Creating HTTP APIs: Incorrect snippet for serialization plugin
In https://ktor.io/docs/server-create-http-apis.html#dependencies:
In order to use
ktor-serialization-kotlinx-json, we also have to apply theplugin.serializationplugin.
However the snippet below shows how to apply the ktor plugin and not the serialization plugin. The snippet is pointing to {src="snippets/tutorial-http-api/build.gradle.kts" include-lines="5,9-10"}. The serialization plugin is also in that file, but at line 8, and it's for an older version of kotlin.
Don't mind contributing a PR for this but not sure if it should point to that file. Pointers welcome (no pun intended).
Timeout topic: Some links lead to 404 pages
On the https://ktor.io/docs/client-timeout.html#configure_plugin page links to requestTimeoutMillis, connectTimeoutMillis and socketTimeoutMillis lead to the 404 page.
Docs: Update Ktor logo and replace favicon
Currently, the JetBrains logo is shown as a favicon on the browser.
We can use the custom-favicons element with a Ktor logo.
Additionally the Ktor logo needs to be updated.
Gradle Plugin
ShadowJar: Support enabling ZIP64 format to overcome limitation of 65535 entries
Please, help me.
When I use the ktor-build-plugin to execute the shadowJar task, the compiler gives me the following error.
> Task :service:service-api:shadowJar FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':service:service-api:shadowJar'.
> shadow.org.apache.tools.zip.Zip64RequiredException: archive contains more than 65535 entries.
To build this archive, please enable the zip64 extension.
See: https://docs.gradle.org/8.5/dsl/org.gradle.api.tasks.bundling.Zip.html#org.gradle.api.tasks.bundling.Zip:zip64
Server
Inconsistent behavior of Netty and rest engines by returning null or empty string for query parameters without values
Use this code to compare between Netty and CIO and visit http://localhost:8080/?auto, http://localhost:8081/?auto`:
fun main() {
embeddedServer(Netty, port = 8080) {
routing {
get("/") {
call.respondText(this.context.request.queryParameters["auto"].toString())
}
}
}.start()
embeddedServer(CIO, port = 8081) {
routing {
get("/") {
call.respondText(this.context.request.queryParameters["auto"].toString())
}
}
}.start(true)
}
CIO sets the query parameter to null (so it's equivalent to auto not being present at all). Netty sets it to a blank string (so you can detect its presence):
cio-qpm.zip
CallLogging, StatusPages: response logged twice when status handler is used
To reproduce, run the following test:
@Test
fun test() = testApplication {
application {
install(CallLogging)
install(StatusPages) {
status(HttpStatusCode.BadRequest) { call, _ ->
call.respondText { "Bad request" }
}
}
routing {
get {
call.respond(HttpStatusCode.BadRequest)
}
}
}
client.get("/")
}
As a result, the response for the / request is unexpectedly logged twice:
10:01:25.843 [DefaultDispatcher-worker-1 @request#2] INFO io.ktor.test -- 200 OK: GET - / in 137ms
10:01:25.843 [DefaultDispatcher-worker-1 @request#2] INFO io.ktor.test -- 200 OK: GET - / in 137ms
IPv6 addresses are not supported in NettyConnectionPoint and CIOConnectionPoint
If you access the server through the IPv6 address, it gives an error
exception: java.lang.NumberFormatException: For input string: "ff:ff:ff:ff:ff:ff:ff]:8080"
at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
at java.base/java.lang.Integer.parseInt(Integer.java:668)
at java.base/java.lang.Integer.parseInt(Integer.java:786)
at io.ktor.server.netty.http1.NettyConnectionPoint.getServerPort(NettyConnectionPoint.kt:55)
at io.ktor.server.routing.HostRouteSelector.evaluate(HostsRoutingBuilder.kt:111)
at io.ktor.server.routing.RoutingResolveContext.handleRoute(RoutingResolveContext.kt:97)
at io.ktor.server.routing.RoutingResolveContext.handleRoute(RoutingResolveContext.kt:157)
at io.ktor.server.routing.RoutingResolveContext.resolve(RoutingResolveContext.kt:82)
at io.ktor.server.routing.Routing.interceptor(Routing.kt:62)
at io.ktor.server.routing.Routing$Plugin$install$1.invokeSuspend(Routing.kt:140)
NettyConnectionPoint contains an error. When trying to get the server port or server host, it splits the string at the first occurrence of the ":" character. But you can access the server using the IPv6 address, so it's better to cut off by the last occurrence of the ":" character (substringAfterLast)
NettyConnectionPoint.kt error place:
override val serverPort: Int
get() = request.headers().get(HttpHeaders.Host)
?.substringAfter(":", defaultPort.toString())
?.toInt()
?: localPort
CIO: File upload fails with `NumberFormatException` when uploading file larger than INT_MAX bytes (~2.1 GiB) since 2.3.0
Tried to upload a large-ish file after recently trying out CIO, now faced with this problem on a 12 GiB file:
an error has happened: For input string: "12255718583"
java.lang.NumberFormatException: For input string: "12255718583"
at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.base/java.lang.Integer.parseInt(Integer.java:652)
at java.base/java.lang.Integer.parseInt(Integer.java:770)
at io.ktor.server.cio.CIOApplicationEngine.hasBody(CIOApplicationEngine.kt:153)
at io.ktor.server.cio.CIOApplicationEngine.access$hasBody(CIOApplicationEngine.kt:24)
at io.ktor.server.cio.CIOApplicationEngine$addHandlerForExpectedHeader$2.invokeSuspend(CIOApplicationEngine.kt:135)
at io.ktor.server.cio.CIOApplicationEngine$addHandlerForExpectedHeader$2.invoke(CIOApplicationEngine.kt)
at io.ktor.server.cio.CIOApplicationEngine$addHandlerForExpectedHeader$2.invoke(CIOApplicationEngine.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.request.ApplicationReceiveFunctionsKt.receiveNullable(ApplicationReceiveFunctions.kt:103)
at io.sebi.ui.AddUploadKt$addUpload$2.invokeSuspend(AddUpload.kt:113)
at io.sebi.ui.AddUploadKt$addUpload$2.invoke(AddUpload.kt)
at io.sebi.ui.AddUploadKt$addUpload$2.invoke(AddUpload.kt)
at io.ktor.server.routing.Route$buildPipeline$1$1.invokeSuspend(Route.kt:116)
at io.ktor.server.routing.Route$buildPipeline$1$1.invoke(Route.kt)
at io.ktor.server.routing.Route$buildPipeline$1$1.invoke(Route.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.server.routing.Routing$executeResult$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
at io.ktor.server.routing.Routing.executeResult(Routing.kt:190)
at io.ktor.server.routing.Routing.interceptor(Routing.kt:64)
at io.ktor.server.routing.Routing$Plugin$install$1.invokeSuspend(Routing.kt:140)
at io.ktor.server.routing.Routing$Plugin$install$1.invoke(Routing.kt)
at io.ktor.server.routing.Routing$Plugin$install$1.invoke(Routing.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invokeSuspend(BaseApplicationEngine.kt:124)
at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invoke(BaseApplicationEngine.kt)
at io.ktor.server.engine.BaseApplicationEngineKt$installDefaultTransformationChecker$1.invoke(BaseApplicationEngine.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.server.application.hooks.CallFailed$install$1$1.invokeSuspend(CommonHooks.kt:45)
at io.ktor.server.application.hooks.CallFailed$install$1$1.invoke(CommonHooks.kt)
at io.ktor.server.application.hooks.CallFailed$install$1$1.invoke(CommonHooks.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:61)
at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:261)
at io.ktor.server.application.hooks.CallFailed$install$1.invokeSuspend(CommonHooks.kt:44)
at io.ktor.server.application.hooks.CallFailed$install$1.invoke(CommonHooks.kt)
at io.ktor.server.application.hooks.CallFailed$install$1.invoke(CommonHooks.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invokeSuspend(DefaultEnginePipeline.kt:118)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invoke(DefaultEnginePipeline.kt)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$1.invoke(DefaultEnginePipeline.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:120)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:78)
at io.ktor.util.pipeline.SuspendFunctionGun.execute$ktor_utils(SuspendFunctionGun.kt:98)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invokeSuspend(Pipeline.kt:478)
at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2$invokeSuspend$$inlined$execute$1.invoke(Pipeline.kt)
at io.ktor.util.debug.ContextUtilsKt.initContextInDebugMode(ContextUtils.kt:17)
at io.ktor.server.cio.CIOApplicationEngine$handleRequest$2.invokeSuspend(CIOApplicationEngine.kt:238)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:111)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:99)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:585)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:802)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:706)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:693)
CallLogging: the plugin completely overrides MDC if at least one entry is configured
To reproduce make a / request to the following server:
embeddedServer(Netty, port = 3000) {
install(CallLogging) {
mdc("hardcoded") {
"1"
}
}
routing {
val r = route("/") {
handle {
call.respondText { "OK" }
}
}
r.intercept(ApplicationCallPipeline.Setup) {
MDC.put("name", "<name>")
withContext(MDCContext()) {
proceed()
}
}
}
}.start(wait = true)
logback.xml:
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>[%X{name}] %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} -%kvp- %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="STDOUT" />
</root>
</configuration>
Expected: the MDC key name has value <name> in the logs.
Actual: the MDC key name has an empty value in the logs.
SSE plugin: Duplicated "Content-Type: text/event-stream" headers
Responses from SSE server contains duplicate headers:
HTTP/1.1 200 OK
Vary: Origin
Content-Type: text/event-stream <-------
Cache-Control: no-store
Connection: keep-alive
Content-Type: text/event-stream <-------
transfer-encoding: chunked
This causes the following exception in the JS client:
SSEException: Expected Content-Type text/event-stream but was: text/event-stream, text/event-stream
This comes from installing the SSE plugin on the client/server and sending an sse response as follows:
sse {
send("Hello")
}
2.3.9
released 4th March 2024
Client
ContentNegotiation: the plugin appends duplicated MIME type to Accept header
If I add a header("Accept", "application/json") at request builder block, the content negotiation plugin will add another one，then the header became "Accept": "application/json,application/json"
Generator
Project generator website not reporting created projects metrics
Since March, we haven't been receiving project created metrics from start.ktor.io. This is likely caused from the front-end library updates.
Server
Allow to set secure cookie even with http scheme
Please consider removing the following exception in ResponseCookies :
if (item.secure && !secureTransport) {
throw IllegalArgumentException("You should set secure cookie only via secure transport (HTTPS)")
}
When using a reverse proxy we can't set the secure cookie. I know there is the XForwardedHeaderSupport plugin but it is not customizable (https://youtrack.jetbrains.com/issue/KTOR-2657) and not all configurations will match the specification described in XForwardedHeaderSupport.
The web framework should let the developper decide if they want to send a secure cookie or not.
In the meantime, here is a hotfix to disable this exception without using XForwardedHeaderSupport:
intercept(ApplicationCallPipeline.Features) {
val endpoint = call.attributes.computeIfAbsent(MutableOriginConnectionPointKey) {
MutableOriginConnectionPoint(call.request.origin)
}
endpoint.scheme = "https"
}