Changelog 1.4 version
1.4.3
released 1st December 2020
Client
Client: NPE in FormDataContentKt -> Input.copyTo
I've received the following exception report for an Android app.
It was sporadic and I cannot reproduce the problem. The problem seems to lie within Ktor itself so I'll simply share what I can.
FATAL EXCEPTION: k.a0 Dispatcher = okhttp3.OkHttpClient Dispatcher
java.lang.NullPointerException
at h.b.a.g.j.b$a.l(Unknown Source:58) = io.ktor.client.request.forms.FormDataContentKt$copyTo$2.invokeSuspend(java.lang.Object)
at kotlin.c0.k.a.a.n(Unknown Source:9) = kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(java.lang.Object)
at kotlinx.coroutines.a1.run(Unknown Source:93)
at kotlinx.coroutines.d3.b.b0(Unknown Source:0)
at kotlinx.coroutines.d3.b$a.c(Unknown Source:14)
at kotlinx.coroutines.d3.b$a.m(Unknown Source:28)
at kotlinx.coroutines.d3.b$a.run(Unknown Source:0)
This is the relevant mapping:
io.ktor.client.request.forms.FormDataContentKt$copyTo$2 -> h.b.a.g.j.b$a:
io.ktor.utils.io.WriterSuspendSession p$ -> p
java.lang.Object L$0 -> q
io.ktor.utils.io.core.Input $this_copyTo -> s
int label -> r
java.lang.Object invoke(java.lang.Object,java.lang.Object) -> M
kotlin.coroutines.Continuation create(java.lang.Object,kotlin.coroutines.Continuation) -> d
java.lang.Object invokeSuspend(java.lang.Object) -> l
The relevant code from Ktor:
https://github.com/ktorio/ktor/blob/a6d68420ee0b272cd09d235057427a6601736487/ktor-client/ktor-client-core/common/src/io/ktor/client/request/forms/FormDataContent.kt#L154
private suspend fun Input.copyTo(channel: ByteWriteChannel) {
if (this is ByteReadPacket) {
channel.writePacket(this)
return
}
channel.writeSuspendSession {
while (!this@copyTo.endOfInput) {
tryAwait(1)
val buffer = request(1)!! // <-- likely cause
val size = this@copyTo.readAvailable(buffer)
if (size < 0) continue
written(size)
}
}
}
CIO: client engine exceptions are both logged and thrown
In the CIO client engine exceptions like SocketTimeoutException
are logged to standard error by the coroutines' uncaught exception handler and thrown by client.get()
.
import io.ktor.client.*
import io.ktor.client.engine.cio.*
import io.ktor.client.features.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import kotlinx.coroutines.*
suspend fun main(): Unit = withContext(Dispatchers.Default) {
HttpClient(CIO) {
install(HttpTimeout) {
socketTimeoutMillis = 5
}
}.use { client ->
try {
client.get<HttpResponse>("http://www.google.com").readText()
} catch (e: Throwable) {
println("Caught: $e")
}
}
}
Output:
Exception in thread "kotlinx.coroutines.DefaultExecutor" java.net.SocketTimeoutException
at io.ktor.network.util.UtilsKt$withSocketTimeout$2.invokeSuspend(Utils.kt:23)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.internal.ScopeCoroutine.afterResume(Scopes.kt:32)
at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:113)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at kotlinx.coroutines.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:188)
at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:122)
at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:111)
at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:308)
at kotlinx.coroutines.CancellableContinuationImpl.cancel(CancellableContinuationImpl.kt:177)
at kotlinx.coroutines.CancellableContinuationImpl.parentCancelled$kotlinx_coroutines_core(CancellableContinuationImpl.kt:184)
at kotlinx.coroutines.ChildContinuation.invoke(JobSupport.kt:1484)
at kotlinx.coroutines.JobSupport.notifyCancelling(JobSupport.kt:1511)
at kotlinx.coroutines.JobSupport.tryMakeCancelling(JobSupport.kt:792)
at kotlinx.coroutines.JobSupport.makeCancelling(JobSupport.kt:752)
at kotlinx.coroutines.JobSupport.cancelImpl$kotlinx_coroutines_core(JobSupport.kt:668)
at kotlinx.coroutines.JobSupport.cancelCoroutine(JobSupport.kt:655)
at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:128)
at kotlinx.coroutines.EventLoopImplBase$DelayedRunnableTask.run(EventLoop.common.kt:497)
at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
at kotlinx.coroutines.DefaultExecutor.run(DefaultExecutor.kt:68)
at java.base/java.lang.Thread.run(Thread.java:832)
Caught: io.ktor.network.sockets.SocketTimeoutException: Socket timeout has been expired [url=http://www.google.com/, socket_timeout=5] ms
HttpRedirect feature alters Location header value
Currently HttpRedirect
feature alters Location
header value by escaping query parameters, but it's significant in case of signed URLs verifications.
val httpClient = HttpClient(Apache) {
followRedirects = true
}
val response = httpClient.get<HttpResponse>("https://dl.bintray.com/jetbrains/markdown/org/jetbrains/markdown-jvm/0.2.0.pre-61/markdown-jvm-0.2.0.pre-61.jar")
assertTrue(response.status.isSuccess())
The original response contains Location
like that:
https://akamai.bintray.com/22/225b067044aa56f36590ef56d41e256cd1d0887b176bfdeec123ecccc6057790?__gda__=exp=1604350711~hmac=417cbd5a97b4c499e2cf7e9eae5dfb9ad95b42cb3ff76c5fb0fae70e2a42db9c&...
But Ktor executes URL encode for query parameters and uses the following value:
https://akamai.bintray.com/22/225b067044aa56f36590ef56d41e256cd1d0887b176bfdeec123ecccc6057790?__gda__=exp%3D1604350711~hmac%3D417cbd5a97b4c499e2cf7e9eae5dfb9ad95b42cb3ff76c5fb0fae70e2a42db9c&...
CIO Engine's HttpClient may fail when trying to send large size binary data.
Ktor Version and Engine Used (client or server and name)
Client CIO, 1.4.1
Describe the bug
HttpClient with CIO Engine sometimes sends an invalid multipart/form-data request when it includes large size binary data.
When it comes to invalid requests, it does not seem to contain a trailing multipart/form-data request boundary.
(There doesn't seem to be enough content in the request body.)
java.nio.BufferOverflowException
seems to occur in the following process.
https://github.com/ktorio/ktor/blob/master/ktor-client/ktor-client-core/common/src/io/ktor/client/request/forms/FormDataContent.kt#L123
This behavior is not reproduced in the case of HttpClient with Apache Engine.
To Reproduce
Sample code is below.
@KtorExperimentalAPI
@Suppress("unused") // Referenced in application.conf
@kotlin.jvm.JvmOverloads
fun Application.module(testing: Boolean = false) {
routing {
post("/multipart") {
val multipartData = call.receiveMultipart()
multipartData.forEachPart {
if (it is PartData.FileItem) {
println(
withContext(Dispatchers.IO) {
it.streamProvider().readAllBytes().size
}
)
}
}
call.respondText("test")
}
get("/execute/cio") {
repeat(10) {
request(HttpClient(CIO))
}
call.respondText("success")
}
get("/execute/apache") {
repeat(10) {
request(HttpClient(Apache))
}
call.respondText("success")
}
}
}
suspend fun request(client: HttpClient) {
val body = formData {
val headers = headersOf(
HttpHeaders.ContentDisposition,
ContentDisposition.File
.withParameter(ContentDisposition.Parameters.Name, "file")
.withParameter(ContentDisposition.Parameters.FileName, "test.jpg")
.toString()
)
appendInput("file", headers) {
// dummy 10MB binary file
ByteArray(10_000_000).inputStream().asInput()
}
}
client.use {
it.submitFormWithBinaryData<Unit>(
url = "http://localhost:8080/multipart",
formData = body
)
}
}
(full code is in https://github.com/ma2Cta/ktor-multipart-request-test/blob/main/src/Application.kt )
-
Start the server.
-
Run
$ curl http://localhost:8080/execute/apache
a few times.
10 times 10000000
will be printed as expected.
- Run
$ curl http://localhost:8080/execute/cio
a few times.
You won't get 10000000
10 times, and you'll probably get a number less than 10000000
the first few times
Expected
The correct request is sent even in the case of CIO Engine.
Timeout feature: android engine throws Java's SocketTimeoutException instead of ConnectTimeoutException
While using Ktor client, with timeout feature, the following instruction can throw a java.net.SocketTimeoutException
instead of the expected ConnectTimeoutException
. The stack trace is in the attachments.
PS: The exceptions section in the Timeout feature documentation is outdated.
iOS client fails with CoroutinesInternalError when Logging is used
Creating a simple HttpClient
with the Logging
feature included fails with the attached stack trace. The failure occurs both for real web calls in-app, as well as for a unit test using MockEngine
. Both the test and the in-app call succeed if the logging configuration is removed.
I have a simple reproducer project attached, and also available at https://github.com/russhwolf/Ktor14
Note that my test configuration is using version 1.3.9-native-mt
of kotlinx.coroutines
, because version 1.3.9
leads to KTOR-915 / KTOR-919
kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[EventLoopImpl@36817bc8, Continuation @ $split$lambda-0$<anonymous>_3COROUTINE$2]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
at kotlin.Error#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:14)
at kotlinx.coroutines.CoroutinesInternalError#<init>(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Exceptions.common.kt:28)
at kotlinx.coroutines.DispatchedTask#handleFatalException(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:93)
at kotlinx.coroutines.DispatchedTask#run(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:64)
at kotlinx.coroutines.EventLoopImplBase#processNextEvent(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:274)
at kotlinx.coroutines#runEventLoop(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:80)
at kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:67)
at kotlinx.coroutines#runBlocking(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:50)
at com.example#runBlocking(/Users/russell/IdeaProjects/Ktor14/shared/src/iosTest/kotlin/runBlocking.kt:8)
at com.example#runBlocking$default(/Users/russell/IdeaProjects/Ktor14/shared/src/iosTest/kotlin/runBlocking.kt:7)
at com.example.ApiClientTest#mockWebCall(/Users/russell/IdeaProjects/Ktor14/shared/src/commonTest/kotlin/ApiClientTest.kt:20)
at com.example.$ApiClientTest$test$0.$mockWebCall$FUNCTION_REFERENCE$0.invoke#internal(/Users/russell/IdeaProjects/Ktor14/shared/src/commonTest/kotlin/ApiClientTest.kt:19)
at com.example.$ApiClientTest$test$0.$mockWebCall$FUNCTION_REFERENCE$0.$<bridge-UNNN>invoke(/Users/russell/IdeaProjects/Ktor14/shared/src/commonTest/kotlin/ApiClientTest.kt:19)
at kotlin.native.internal.test.BaseClassSuite.TestCase#run(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestSuite.kt:85)
at kotlin.native.internal.test.TestRunner.run#internal(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:236)
at kotlin.native.internal.test.TestRunner.runIteration#internal(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:258)
at kotlin.native.internal.test.TestRunner#run(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:273)
at kotlin.native.internal.test#testLauncherEntryPoint(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:22)
at kotlin.native.internal.test#main(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:26)
at <global>.Konan_start(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:25)
at <global>.Init_and_run_start(Unknown Source)
at <global>.start(Unknown Source)
at kotlin.Exception#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:25)
at kotlin.RuntimeException#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:36)
at kotlinx.coroutines.CompletionHandlerException#<init>(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Exceptions.common.kt:13)
at kotlinx.coroutines.JobSupport.notifyCompletion#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:365)
at kotlinx.coroutines.JobSupport.completeStateFinalization#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:326)
at kotlinx.coroutines.JobSupport.finalizeFinishingState#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:241)
at kotlinx.coroutines.JobSupport.tryMakeCompletingSlowPath#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:922)
at kotlinx.coroutines.JobSupport.tryMakeCompleting#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:875)
at kotlinx.coroutines.JobSupport#makeCompletingOnce(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:840)
at kotlinx.coroutines.AbstractCoroutine#resumeWith(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/AbstractCoroutine.kt:111)
at kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:43)
at kotlinx.coroutines.DispatchedTask#run(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:53)
at kotlinx.coroutines.EventLoopImplBase#processNextEvent(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:274)
at kotlin.Throwable#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Throwable.kt:23)
at kotlin.Exception#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:23)
at kotlin.RuntimeException#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:34)
at kotlin.native.concurrent.InvalidMutabilityException#<init>(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/concurrent/Freezing.kt:22)
at <global>.ThrowInvalidMutabilityException(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/concurrent/Internal.kt:92)
at <global>.MutationCheck(Unknown Source)
at kotlinx.coroutines.AwaitAll.AwaitAllNode.<set-handle>#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Await.kt:94)
at kotlinx.coroutines.AwaitAll.await#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Await.kt:68)
at kotlinx.coroutines.$awaitAllCOROUTINE$1#invokeSuspend(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Await.kt:38)
at kotlinx.coroutines#awaitAll@kotlin.collections.Collection<kotlinx.coroutines.Deferred<0:0>>(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Await.kt:37)
at io.ktor.util.$split$lambda-0COROUTINE$3.invokeSuspend#internal(/opt/buildAgent/work/a85294440dc5c6e/ktor-utils/common/src/io/ktor/util/ByteChannels.kt:29)
at kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:30)
at kotlinx.coroutines.DispatchedTask#run(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:39)
at kotlinx.coroutines.EventLoopImplBase#processNextEvent(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:274)
at kotlinx.coroutines#runEventLoop(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:80)
at kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:67)
at kotlinx.coroutines#runBlocking(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:50)
at com.example#runBlocking(/Users/russell/IdeaProjects/Ktor14/shared/src/iosTest/kotlin/runBlocking.kt:8)
at com.example#runBlocking$default(/Users/russell/IdeaProjects/Ktor14/shared/src/iosTest/kotlin/runBlocking.kt:7)
at com.example.ApiClientTest#mockWebCall(/Users/russell/IdeaProjects/Ktor14/shared/src/commonTest/kotlin/ApiClientTest.kt:20)
at com.example.$ApiClientTest$test$0.$mockWebCall$FUNCTION_REFERENCE$0.invoke#internal(/Users/russell/IdeaProjects/Ktor14/shared/src/commonTest/kotlin/ApiClientTest.kt:19)
at com.example.$ApiClientTest$test$0.$mockWebCall$FUNCTION_REFERENCE$0.$<bridge-UNNN>invoke(/Users/russell/IdeaProjects/Ktor14/shared/src/commonTest/kotlin/ApiClientTest.kt:19)
at kotlin.native.internal.test.BaseClassSuite.TestCase#run(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestSuite.kt:85)
at kotlin.native.internal.test.TestRunner.run#internal(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:236)
at kotlin.native.internal.test.TestRunner.runIteration#internal(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:258)
at kotlin.native.internal.test.TestRunner#run(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/TestRunner.kt:273)
at kotlin.native.internal.test#testLauncherEntryPoint(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:22)
at kotlin.native.internal.test#main(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:26)
at <global>.Konan_start(/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/internal/test/Launcher.kt:25)
at <global>.Init_and_run_start(Unknown Source)
Core
Input.readTextExactBytes(n) on empty input different behavior per platform
On JVM, if Input
is empty, it will throw EOFException
On Native/JS it will just return empty string
reproducer:
println(ByteReadPacket.Empty.readTextExactBytes(1).length)
JVM: fail with EOF
JS/Native: print 0
ktor-io: JVM shared function decrease performance starting from 1.4.0
While profiling rsocket-kotlin
I found in graphs/traces that one of the most frequent element are calls to SharedJVM
and boxing of Int
values. I know, that shared
delegate used in ktor-io
to support using atomics on K/N and plain value on JVM/JS. But that code optimization
isn't ok regarding to performance.
The issue consists of two parts:
- every usage of
shared
- create anonymous object on initialization - boxing/unboxing of primitives on setting/getting value
F.e.Buffer
class fromktor-io
useshared
for 4Int
fields:readPosition
,writePosition
,startGap
,limit
- and all of them are updating frequently. So all time we operate on them we will do box/unbox - performance suffer significantly...
To proof my idea, I've replaced calls of shared
to plain variables inside ktor-io
module, and built it locally. Running benchmarks I found, that it's 50% faster!
So, f.e. io.rsocket.kotlin.benchmarks.RSocketBenchmark#requestResponseParallel
benchmark on ktor version 1.4.2
gives ~200 ops/s and on local version without shared gives ~300 ops/s.
That benchmark heavily use big ByteReadPacket
s.
I think that should be fixed some how, as 50% difference is really big
Experimental API and compatibility guarantees
Problem
Currently, we have a public commitment to not break any API and ABI. This is generally good.
The problem is that it makes it almost impossible for a new experimental feature/API to grow and entirely stops evolution since major versions are released too slowly.
Proposed solution
The idea is to relax guarantees for all API that is marked with experimental annotations similar to Kotlin.
Rules
- in a patch version: no changes allowed, deprecations with level WARNING could be introduced for existing API.
- in a minor version: both source and binary changes allowed except for complete removal. All changes should include migration/deprecation. Hidden deprecated APIs should be introduced for binary compatibility.
- in a major version: any changes including complete removal but only through the deprecation cycle.
Requirements for experimental API:
- experimental API should be marked with an experimental annotation requiring opt-in
- an experimental annotation should have level ERROR and should have a reasonable and clear description
- for a single or group of new experimental functions/classes/APIs having the same purpose and/or related functionality, it is required to introduce a new experimental annotation
Note: the consequence is that you can't use KtorExperimentalAPI
annotation for new API. So one should always create new experimental annotations for now.
The other consequence is that we should revisit all KtorExperimentalAPI usages and either stabilize them or migrate to new experimental annotations.
Requirements for experimental API annotation:
An experimental annotation class should:
- require opt-in with level ERROR (
@RequiresOptIn(level = ERROR)
); - have a reasonable and clear description;
- have a reference to a ktor ticket that introduced this API and/or describes why the API is experimental and that is useful for tracking the concrete experimental feature status;
- have retention
BINARY
, unless there is a reason to haveRUNTIME
; - be documented;
- contain a disclamer in the kdoc about the risk of using experimental features.
Note: RUNTIME
retention is not recommended since the ability to analyze it in runtime makes it more difficult to remove the annotation due to indirect and unobvious references. Retention SOURCE
is not acceptable because it is how @RequiresOptIn
works in Kotlin.
Experimental API Deprecation cycle
A deprecation cycle for an API that is intended to be removed or changed consist of the following steps:
- In any release, make the API deprecated with level WARNING by applying one of the following:
@Deprecated
annotation with level WARNING, provide a clear message describing what will be changed and what exact migration is required, provide a "replace with"- a new annotation requiring opt-in with level WARNING and the description
- In the next minor release, promote the deprecation level from WARNING to ERROR
- In the next minor release after step 2, change the deprecation level from ERROR to HIDDEN, keep the implementation working despite that it is hidden (to keep binary compatibility)
- In the next major release, remove the old API
Please note that using an opt-in annotation for deprecation doesn't provide the ability to write a "replace with". Thus, it only makes sense to use it for removing old API. A good example is kotlinx.coroutines ObsoleteCoroutinesApi
If some of the rules are impossible to comply (for example, it is impossible to provide a "replace with" action or impossible to keep a hidden function working), an author of change should initiate a discussion at least with PR reviewer. When decided, it could be also discussed in the team or even publically for a quite popular experimental API. If it was decided to skip some of the requirements, it is important to document that decision and reasons in the PR.
KtorExperimentalAPI deprecation
@KtorExperimentalAPI
annotation is going to be deprecated in the following stages:
- Revisit all existing API marked with the annotation and stabilize when possible without breaking changes in a minor version(s). Mark the annotation deprecated with level WARNING.
- In major version (2.0.0), revisit again. When impossible to stabilize, introduce new experimental annotations and use them instead. Ensure no KtorExperimentalAPI usages. Promote its deprecation to level ERROR.
- In the next major version (3.0.0) deprecate it with level HIDDEN. Use text search to ensure no text occurrences.
- In the next major version (4.0.0) remove it.
The reason why we deprecate it is that it introduces a way of "anonymous" experiment. An experimental API should be introduced with a clear reason and plan. A plan should cover only one particular feature/functionality. An experimental feature should have a more or less clear lifecycle.
The other disadvantage of this annotation is that there is no easy way to de-opt-in users from it. Also, it invites most brave users to opt-in globally for the whole project (just as we did for ExperimentalCoroutinesApi
). This is very bad because in this case experimental usages will become uncontrolled and the user doesn't realize anymore what danger it is. So to prevent such problems, we should stop using it and migrate to a better approach.
Example
Here is an example of an experimental API:
/**
* API marked with this annotation is a part of experimental Z feature.
* The feature preview provides the ability to work with Z.
* Please see KTOR-number for details and stabilization plan.
*
* This is an API preview and it's not recommended for production development
* since it could be changed or removed in the future and
* not guaranteed to remain functional and its behaviour may also change in any non-patch release.
*/
@RequiresOptIn("Z feature is experimental and not recommended for production development.", level = ERROR)
@Retention(BINARY)
annotation class ZFeature
/**
* Z feature's context
*/
@ZFeature
class ZContext
/**
* Configures Z feature and invokes [block] with running instance.
*/
@ZFeature
fun withZ(block: () -> Unit) = ...
Infrastructure
Add build parameter to build ktor with JVM IR compiler
From PR #2199:
To be used in specific kotlinx-train CI configuration. It's needed for faster adoption of JVM IR compiler.
CI does not trigger build on PR to samples repo
PRs to https://github.com/ktorio/ktor-samples are not checked by CI
Invalid publication 'kotlinMultiplatform' for task :publishKotlinMultiplatformPublicationToMavenLocal on master
To reproduce run ./gradlew :publishKotlinMultiplatformPublicationToMavenLocal
in the root directory of the Ktor framework repository using the latest master.
> Failed to publish publication 'kotlinMultiplatform' to repository 'mavenLocal'
> Invalid publication 'kotlinMultiplatform': multiple artifacts with the identical extension and classifier ('jar', 'null').
OS: macOS Catalina
Server
Request headers exceeding expected threshold are not handled correctly
Kotlin version: 1.3.70
Ktor version: 1.3.2
In situations when receiving large cookies and the headers exceed the 8K limit (using Netty as engine), not all headers get loaded.
Example 1:
- When we have the
ContentNegotiation
feature enabled - And a
content-type: application/json
is present on the request - And the cookie header is very large (the sum of all headers exceeding 8K)
- Then the
ContentNegotiation
feature resolves the content type toAny
(*/*
) - probably because it can't load all headers - This results in a situation where the server responds with
415 - Unsupported Media Type
.
In this particular example probably a 431 - Requests header fields too large
is more appropriate.
Example 2:
- Consider the hypothesis from Example 1
- When the
content-type: application/json
header is loaded before the cookie header - Then the request passes successfully through the
ContentNegotiation
feature - But the request hangs
This seems like an issue where Ktor does not correctly interpret errors from the Netty engine.
Sample engine error:
DefaultHttpRequest(decodeResult: failure(io.netty.handler.codec.TooLongFrameException: HTTP header is larger than 8192 bytes.), version: HTTP/1.1)
POST /v1/api HTTP/1.1
Host: localhost:8080
Accept: */*
cache-control: no-cache
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36
content-type: text/plain;charset=UTF-8
Upgrade Netty to 4.1.54.Final
"Wrong HEX escape": gracefully handle invalid URLs
We recently got some alerts in our webserver because ktor threw an uncaught exception. Ideally these are handled and a 400 is sent back.
Here is the exception:
Wrong HEX escape: %uf, in /%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/%uff0e%uff0e/winnt/win.ini
Stacktrace:
at io.ktor.http.CodecsKt.decodeImpl (Codecs.kt:197)
at io.ktor.http.CodecsKt.decodeScan (Codecs.kt:146)
at io.ktor.http.CodecsKt.decodeURLPart (Codecs.kt:140)
at io.ktor.http.CodecsKt.decodeURLPart$default (Codecs.kt:139)
at io.ktor.routing.RoutingResolveContext.parse (RoutingResolve.kt:70)
at io.ktor.routing.RoutingResolveContext.<init> (RoutingResolve.kt:49)
at io.ktor.routing.Routing.interceptor (Routing.kt:31)
at io.ktor.routing.Routing$Feature$install$1.invokeSuspend (Routing.kt:99)
at io.ktor.routing.Routing$Feature$install$1.invoke (Routing.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop (PipelineContext.kt:318)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed (PipelineContext.kt:163)
at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend (ContentNegotiation.kt:107)
at io.ktor.features.ContentNegotiation$Feature$install$1.invoke (ContentNegotiation.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop (PipelineContext.kt:318)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed (PipelineContext.kt:163)
at io.ktor.util.pipeline.SuspendFunctionGun.execute (PipelineContext.kt:183)
at io.ktor.util.pipeline.Pipeline.execute (Pipeline.kt:27)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend (DefaultEnginePipeline.kt:120)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke (DefaultEnginePipeline.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop (PipelineContext.kt:318)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed (PipelineContext.kt:163)
at io.ktor.util.pipeline.SuspendFunctionGun.execute (PipelineContext.kt:183)
at io.ktor.util.pipeline.Pipeline.execute (Pipeline.kt:27)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend (NettyApplicationCallHandler.kt:40)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke (NettyApplicationCallHandler.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched (Undispatched.kt:55)
at kotlinx.coroutines.CoroutineStart.invoke (CoroutineStart.kt:111)
at kotlinx.coroutines.AbstractCoroutine.start (AbstractCoroutine.kt:158)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch (Builders.common.kt:54)
at kotlinx.coroutines.BuildersKt.launch (Unknown Source)
at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest (NettyApplicationCallHandler.kt:30)
at io.ktor.server.netty.NettyApplicationCallHandler.channelRead (NettyApplicationCallHandler.kt:24)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead (AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.access$600 (AbstractChannelHandlerContext.java:61)
at io.netty.channel.AbstractChannelHandlerContext$7.run (AbstractChannelHandlerContext.java:370)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute (AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks (SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run (NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run (SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run (ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run (FastThreadLocalRunnable.java:30)
Netty HTTP/2 HEAD response hangs
HttpServerTestSuite.testHeadResponse
fails with Netty HTTP/2 because a response message has no actual body but headers have the concrete content length so it leads to the pipeline failure
Sessions + SSL (Netty)
edit: https://github.com/dm6801/ktor_sessions_ssl
when trying to use Session feature with cookies
install(Sessions) {
cookie<MySession>("MY_SESSION")
}
accessing https:/localhost:443/ throws:
java.lang.UnsupportedOperationException: null
(Coroutine boundary)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:55)
(Coroutine creation stacktrace)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:188)
at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:38)
at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:29)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.ktor.server.netty.EventLoopGroupProxy$Companion$create$factory$1$1.run(NettyApplicationEngine.kt:215)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:821)
Caused by: java.lang.UnsupportedOperationException: null
at io.ktor.server.netty.http2.NettyHttp2ApplicationRequest.getCookies(NettyHttp2ApplicationRequest.kt:64)
at io.ktor.sessions.SessionTransportCookie.receive(SessionTransportCookie.kt:28)
at io.ktor.sessions.SessionsKt.receiveSessionData(Sessions.kt:204)
at io.ktor.sessions.Sessions$Feature$install$1.invokeSuspend(Sessions.kt:59)
at io.ktor.sessions.Sessions$Feature$install$1.invoke(Sessions.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
relevant line NettyHttp2ApplicationRequest.getCookies(NettyHttp2ApplicationRequest.kt:64):
override val cookies: RequestCookies
get() = throw UnsupportedOperationException()
- http://localhost:8080/ - works
Using blocking primitives on this dispatcher is not allowed. Consider using async channel instead or use blocking primitives in withContext(Dispatchers.IO) instead.
failing on: val bookRequest = call.receive<Example>()
(full example):
package app.controllers
import app.models.Example
import app.services.ExampleService
import io.ktor.application.*
import io.ktor.auth.*
import io.ktor.auth.jwt.*
import io.ktor.features.*
import io.ktor.http.*
import io.ktor.request.*
import io.ktor.response.*
import io.ktor.routing.*
import org.kodein.di.instance
class ExampleController(application: Application): Controller(application) {
private val exampleService: ExampleService by instance()
override fun Route.getRoutes() {
post("/") {
val bookRequest = call.receive<Example>()
exampleService.addExample(bookRequest)
call.respond(HttpStatusCode.Accepted)
}
}
}
Other
Remove copyTo usage from ServerPipeline
Handle failure in reading request body
@Test
fun testErrorInWritingPropagates() = testSuspend {
val client = HttpClient(factory)
val channel = ByteChannel(true)
channel.writeAvailable("text".toByteArray())
channel.close(IllegalStateException("Error on write"))
assertFailsWith<IllegalStateException>("Error on write") {
val result = client.post<String>("http://localhost:$serverPort/echo") {
body = channel
}
}
}
Apache: sends empty body without error
OkHttp: hangs
CIO: sends empty body without error
Android: succeed
`ByteBufferChannel.readRemaining` doesn't read whole channel
Client: URL encode / escaping is wrong
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1945
Ktor Version and Engine Used (client or server and name)
Version: 1.3.2-1.4-M1-2
ktor-client-core
ktor-client-json-native
ktor-client-serialization-native
ktor-client-curl
Describe the bug
The URL is wrong escaped.
To Reproduce
class FahrplanApi(private val serverUrl: String = "http://api.deutschebahn.com/freeplan/v1") {
private val client = HttpClient {
install(JsonFeature)
// workaround for version 1.4-M1
// https://github.com/ktorio/ktor/issues/1820#issuecomment-633519483
HttpResponseValidator {
validateResponse { response ->
when (response.status.value) {
in 300..399 -> throw RedirectResponseException(response)
in 400..499 -> throw ClientRequestException(response)
in 500..599 -> throw ServerResponseException(response)
}
}
}
expectSuccess = false
}
suspend fun departureBoardDateTime(id: Long, date: String) =
client.get<Departure>("$serverUrl/departureBoard/$id?date=$date")
}
fun main(): Unit = runBlocking {
val api = FahrplanApi()
api.departureBoardDateTime(8000096, "2020-06-14T20:21:22")
}
after run:
Uncaught Kotlin exception: io.ktor.client.features.ClientRequestException: Client request(https://api.deutschebahn.com/freeplan/v1/departureBoard/8000096?date=2020-06-14T20%253A21%253A22) invalid: 400 Bad Request
Expected behavior
https://api.deutschebahn.com/freeplan/v1/departureBoard/8000096?date=2020-06-14T20%3A21%3A22
%253A
--> %3A
(25 is actually %
)
Fix configuration if project without VPN and cache
Update kotlin to 1.4.20
Wrong pool is used to release `IOBuffer` after `ByteChannelSequential.copyTo` from static initialized instance.
1.4.2
released 10th November 2020
Client
Ktor 1.4.1 fails to upload file (submit form) when logging level = LogLevel.BODY
I am using ktor 1.4.1 in my multiplatform project to submit a file.
the weird thing is that whenever I change the log level to BODY the form parameters are corrupt, but when it is INFO it works correctly.
install(Logging) {
logger = object : Logger {
override fun log(message: String) {
println(message)
}
}
level = LogLevel.BODY // FAILS to submit
level = LogLevel.INFO // OK
}
here is the code to submit file
httpClient.submitFormWithBinaryData(
"https://myendpoint",
formData {
appendInput(
key = "file",
headers = Headers.build {
append(ContentDisposition, " filename=video.mp4")
}
) { buildPacket { writeFully(data) } }
}
)
And when I tried to debug, I found out that when level = LogLevel.BODY
ktor does not send
Content-Type: multipart/form-data; boundary=XXX
in request. It is only available in logs but not actually sent. Maybe logger consumes it or sth like this happens.
"IllegalStateException: Cannot execute task because event loop was shut down" for native client
To reproduce run ./gradlew runDebugExecutableMacosX64
in the root directory of the attached project.
As a result, I get the following unexpected exception:
Uncaught Kotlin exception: kotlin.IllegalStateException: Cannot execute task because event loop was shut down
at 0 loop_shutdown_coroutines.kexe 0x000000010b88270d kfun:kotlin.Throwable#<init>(kotlin.String?){} + 93 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Throwable.kt:23:37)
at 1 loop_shutdown_coroutines.kexe 0x000000010b87ba7b kfun:kotlin.Exception#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
at 2 loop_shutdown_coroutines.kexe 0x000000010b87bc3b kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
at 3 loop_shutdown_coroutines.kexe 0x000000010b87c14b kfun:kotlin.IllegalStateException#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:70:44)
at 4 loop_shutdown_coroutines.kexe 0x000000010ba5639e kfun:kotlinx.coroutines#loopWasShutDown(){}kotlin.Nothing + 222 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/CoroutineContext.kt:25:50)
at 5 loop_shutdown_coroutines.kexe 0x000000010ba5887d kfun:kotlinx.coroutines.EventLoopImplPlatform#reschedule(kotlin.Long;kotlinx.coroutines.EventLoopImplBase.DelayedTask){} + 61 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/EventLoop.kt:22:9)
at 6 loop_shutdown_coroutines.kexe 0x000000010b9e657b kfun:kotlinx.coroutines.EventLoopImplBase.rescheduleAllDelayed#internal + 587 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:397:13)
at 7 loop_shutdown_coroutines.kexe 0x000000010b9e2dde kfun:kotlinx.coroutines.EventLoopImplBase#shutdown(){} + 270 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:226:9)
at 8 loop_shutdown_coroutines.kexe 0x000000010ba58d1c kfun:kotlinx.coroutines.EventLoopImpl#shutdown(){} + 140 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/EventLoop.kt:34:15)
at 9 loop_shutdown_coroutines.kexe 0x000000010b9e12a4 kfun:kotlinx.coroutines.EventLoop#decrementUseCount(kotlin.Boolean){} + 324 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:114:13)
at 10 loop_shutdown_coroutines.kexe 0x000000010b9e137c kfun:kotlinx.coroutines.EventLoop#decrementUseCount$default(kotlin.Boolean;kotlin.Int){} + 124 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:108:5)
at 11 loop_shutdown_coroutines.kexe 0x000000010ba5383b kfun:kotlinx.coroutines#runEventLoop(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>){} + 1115 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:85:20)
at 12 loop_shutdown_coroutines.kexe 0x000000010ba551f0 kfun:kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal + 464 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:67:9)
at 13 loop_shutdown_coroutines.kexe 0x000000010ba52dcf kfun:kotlinx.coroutines#runBlocking(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,0:0>){0§<kotlin.Any?>}0:0 + 1311 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:50:22)
at 14 loop_shutdown_coroutines.kexe 0x000000010ba5330c kfun:kotlinx.coroutines#runBlocking$default(kotlin.coroutines.CoroutineContext?;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,0:0>;kotlin.Int){0§<kotlin.Any?>}0:0 + 348 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:33:15)
at 15 loop_shutdown_coroutines.kexe 0x000000010b82bc48 kfun:#main(){} + 168 (/Users/Aleksei.Tirman/projects/jb/loop_shutdown_coroutines/src/macosX64Main/kotlin/App.kt:6:14)
at 16 loop_shutdown_coroutines.kexe 0x000000010b82c5c0 Konan_start + 144 (/Users/Aleksei.Tirman/projects/jb/loop_shutdown_coroutines/src/macosX64Main/kotlin/App.kt:6:1)
at 17 loop_shutdown_coroutines.kexe 0x000000010b831f4b Init_and_run_start + 107
at 18 libdyld.dylib 0x00007fff6a1c9cc9 start + 1
Unfinished workers detected, 1 workers leaked!
Use `Platform.isMemoryLeakCheckerActive = false` to avoid this check.
Abort trap: 6
Here is the similar issue.
I've reproduced this problem for both macOS and Linux targets.
Kotlin multiplatform: 1.4.10
Kotlin coroutines: 1.3.9-native-mt-2
Ktor: 1.4.0
Overload resolution ambiguity: HttpClient.ws
import io.ktor.client.*
import io.ktor.client.engine.cio.*
import io.ktor.client.features.websocket.*
suspend fun main() {
HttpClient(CIO).ws("http://localhost") {}
}
Overload resolution ambiguity:
public suspend fun HttpClient.ws(urlString: String, request: HttpRequestBuilder.() -> Unit = ..., block: suspend DefaultClientWebSocketSession.() -> Unit): Unit defined in io.ktor.client.features.websocket
public suspend fun HttpClient.ws(urlString: String, request: HttpRequestBuilder.() -> Unit = ..., block: suspend DefaultClientWebSocketSession.() -> Unit): Unit defined in io.ktor.client.features.websocket
There's one in core-jvm
and one in cio-jvm
.
Content-type header is missing in Request when using LogLevel.BODY
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1690
Ktor Version and Engine Used (client or server and name)
version: 1.3.1
lib: ktor-client-cio
additional modules: ktor-client-logging-jvm
Describe the bug
Content-type header is missing when LogLevel.BODY
is set. (also I am using multi-part body, not sure if relevant). Header exists if removing logging or setting LogLevel.HEADER
.
To Reproduce
Steps to reproduce the behavior:
- Set logging
HttpClient {
install(Logging) {
logger = Logger.ANDROID
level = LogLevel.BODY
}
}
- Do multi-form request
val response = httpClient.post<String> {
url("http://someurl")
body = MultiPartFormDataContent(formData {
append(
"file",
"file.txt",
ContentType.parse("text/plain"),
contentBytes.size.toLong()
) {
writeFully(contentBytes)
}
})
- Observe missing
Content-Type: multipart/form-data; <...>
header
Expected behavior
Content-Type: multipart/form-data; <...>
in request exists
Jetty client engine with high concurrency randomly freezes, throws NPEs, ISEs or EofExceptions
Exception in thread "main" java.lang.NullPointerException
at java.base/java.util.Objects.requireNonNull(Objects.java:222)
at org.eclipse.jetty.io.SelectorManager.connect(SelectorManager.java:193)
at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:409)
at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:382)
at io.ktor.client.engine.jetty.JettyHttpRequestKt$connect$2.invoke(JettyHttpRequest.kt:61)
at io.ktor.client.engine.jetty.JettyHttpRequestKt$connect$2.invoke(JettyHttpRequest.kt)
at io.ktor.client.engine.jetty.UtilsKt.withPromise(utils.kt:14)
at io.ktor.client.engine.jetty.JettyHttpRequestKt.connect(JettyHttpRequest.kt:59)
at io.ktor.client.engine.jetty.JettyHttpRequestKt.executeRequest(JettyHttpRequest.kt:31)
at io.ktor.client.engine.jetty.JettyHttp2Engine.execute(JettyHttp2Engine.kt:38)
at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:86)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Suppressed: java.lang.NullPointerException
... 17 more
Suppressed: java.lang.NullPointerException
... 17 more
Suppressed: java.lang.NullPointerException
... 17 more
Suppressed: java.lang.NullPointerException
... 17 more
import io.ktor.client.*
import io.ktor.client.engine.jetty.*
import io.ktor.client.features.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import kotlinx.coroutines.*
suspend fun main() = withContext(Dispatchers.Default) {
HttpClient(Jetty) {
expectSuccess = false
install(HttpTimeout) {
connectTimeoutMillis = 2_000
socketTimeoutMillis = 2_000
}
engine {
threadsCount = 200
}
}.use { client ->
coroutineScope {
repeat(5) {
repeat(200) {
launch {
client.get<HttpResponse>("https://esi.evetech.net/latest/markets/10000040/orders/?page=1")
.readText()
.let { println(it.length) }
}
}
}
}
}
println()
println("Done")
}
Native: ResponseObserver hangs coroutine indefinitely
Here's my reproducer:
launch {
try {
val client = HttpClient()
ResponseObserver.install(ResponseObserver {
// Whatever
println("Bonjour!")
}, client)
client.get<HttpResponse>("https://www.google.com/")
onSuccess()
} catch (t: Throwable) {
t.printStackTrace()
onError()
}
}
Reproducer project here: https://github.com/Ribesg/kotlin-reproducer/tree/ktor-1095
Just clone, open in XCode 11.x and Run (tested on simulator).
CertificatePinner buildErrorMessage outputs CertificatesInfo reference and variable name (HASH_ALGORITHM_SHA_256) instead of value
When making the certificate pinner fail on iOS, it outputs this:
Certificate pinning failure!
Peer certificate chain:
io.ktor.client.engine.ios.certificates.CertificatesInfo@3d83458.HASH_ALGORITHM_SHA_256fEbY8nKEFY2ODyxn2P0kyIq+C2/Rkke8ifByvsmamLw=: *.domain.com
io.ktor.client.engine.ios.certificates.CertificatesInfo@3d83458.HASH_ALGORITHM_SHA_256hETpgVvaLC0bvcGG3t0cuqiHvr4XyP2MTwCiqhgRWwU=: GlobalSign RSA OV SSL CA 2018
io.ktor.client.engine.ios.certificates.CertificatesInfo@3d83458.HASH_ALGORITHM_SHA_256cGuxAXyFXFkWm61cF4HPWX8S0srS9j0aSqN0k4AP+4A=: GlobalSign
It inserts the reference to the CertificatesInfo object followed by the name of the variable HASH_ALGORITHM_SHA_256.
Expected output can be read in the documentation of CertificatePinner.
This is a bug in the error message building in CertificatePinner.buildErrorMessage().
Client: InvalidCacheStateException when Vary header differs for 200 and 304 responses
I perform a lot of requests in parallel with relatively short-lived Expires values. That leads to plenty of exceptions like this:
Exception in thread "main" io.ktor.client.features.cache.InvalidCacheStateException: The entry for url: https://esi.evetech.net/latest/markets/10000035/orders/?page=1 was removed from cache
at io.ktor.client.features.cache.HttpCache$Companion$install$2.invokeSuspend(HttpCache.kt:104)
at io.ktor.client.features.cache.HttpCache$Companion$install$2.invoke(HttpCache.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:323)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:168)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:188)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:31)
at io.ktor.client.HttpClient$1.invokeSuspend(HttpClient.kt:141)
at io.ktor.client.HttpClient$1.invoke(HttpClient.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:323)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:168)
at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(PipelineContext.kt:178)
at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:69)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:830)
Ktor Client HTTP Cache for invalid or blank Expires header
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1892
Ktor Version and Engine Used (client or server and name)
1.3.2, client, io.ktor.client.features.cache.HttpCache
Describe the bug
If a server returns a blank or an invalid Expires HTTP header, the ktor client crashes if the feature HttpCache is enabled install(HttpCache)
To Reproduce
Steps to reproduce the behavior:
- Write the following:
val articleUrl = Url("https://www.nachrichten.at/panorama/weltspiegel/30-jahre-mauerfall-sinnbild-der-freiheit;art17,3184503")
val newsClient = HttpClient(Apache) {
install(HttpCache)
install(UserAgent) {
agent = "Mozilla/5.0"
}
}
try {
val call: HttpStatement = newsClient.get(articleSimilarityRequest.articleUrl)
val response: HttpResponse = call.receive()
val requestUrl = response.request.url.toString()
val rawHtml = response.readText()
println(rawHtml)
} catch (cause: Throwable) {
println("Unable to download article")
} finally {
newsClient.close()
}
- If you open the URL
https://www.nachrichten.at/panorama/weltspiegel/30-jahre-mauerfall-sinnbild-der-freiheit;art17,3184503
in the browser, you can see that the Expires response header is blank - An exception is thrown if you run the code:
2020-05-23 14:41:24.124 [DefaultDispatcher-worker-2 @request#48] ERROR a.s.api.util.service.DownloadService - Unable to download article
java.lang.IllegalStateException: Failed to parse date:
at io.ktor.http.DateUtilsKt.fromHttpToGmtDate(DateUtils.kt:40)
at io.ktor.client.features.cache.HttpCacheEntryKt.cacheExpires(HttpCacheEntry.kt:84)
at io.ktor.client.features.cache.HttpCacheEntryKt.HttpCacheEntry(HttpCacheEntry.kt:18)
at io.ktor.client.features.cache.HttpCacheEntryKt$HttpCacheEntry$1.invokeSuspend(HttpCacheEntry.kt)
(Coroutine boundary)
at io.ktor.client.HttpClient$1.invokeSuspend(HttpClient.kt:123)
at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:68)
at io.ktor.client.features.HttpSend$DefaultSender.execute(HttpSend.kt:116)
at io.ktor.client.features.HttpSend$Feature$install$1.invokeSuspend(HttpSend.kt:79)
at io.ktor.client.features.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:89)
at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invokeSuspend(HttpRequestLifecycle.kt:34)
at io.ktor.client.HttpClient.execute(HttpClient.kt:164)
at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:104)
at io.ktor.client.statement.HttpStatement.execute(HttpStatement.kt:43)
at app.shwoo.api.util.service.DownloadService.downloadArticle$suspendImpl(DownloadService.kt:101)
at app.shwoo.api.server.web.SimilarityResourceKt$similarity$1$4.invokeSuspend(SimilarityResource.kt:38)
at io.ktor.routing.Routing.executeResult(Routing.kt:147)
at io.ktor.routing.Routing.interceptor(Routing.kt:34)
at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:99)
at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:107)
at io.ktor.server.testing.TestApplicationEngine$callInterceptor$1.invokeSuspend(TestApplicationEngine.kt:292)
at io.ktor.server.testing.TestApplicationEngine$2.invokeSuspend(TestApplicationEngine.kt:50)
at io.ktor.server.testing.TestApplicationEngine$handleRequest$pipelineJob$1.invokeSuspend(TestApplicationEngine.kt:290)
Caused by: java.lang.IllegalStateException: Failed to parse date:
at io.ktor.http.DateUtilsKt.fromHttpToGmtDate(DateUtils.kt:40)
at io.ktor.client.features.cache.HttpCacheEntryKt.cacheExpires(HttpCacheEntry.kt:84)
at io.ktor.client.features.cache.HttpCacheEntryKt.HttpCacheEntry(HttpCacheEntry.kt:18)
at io.ktor.client.features.cache.HttpCacheEntryKt$HttpCacheEntry$1.invokeSuspend(HttpCacheEntry.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Expected behavior
It seems that the problem occurs in https://github.com/ktorio/ktor/blob/master/ktor-client/ktor-client-core/common/src/io/ktor/client/features/cache/HttpCacheEntry.kt#L84. If the Expires header is blank an exception is thrown. The expected behavior is that the application handles the cache as expired if an invalid date is provided in the HTTP Expires header. Like described in https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expires
NoTransformationFoundException when using JsonFeature and connecting through a proxy server
Steps of reproduction:
- Add windows or macOS targets to the Gradle build file if required.
- Run
./gradlew runDebugExecutableLinuxX64
(replace with your target)
As a result, the following exception is thrown:
> Task :runDebugExecutableLinuxX64 FAILED
Uncaught Kotlin exception: io.ktor.client.call.NoTransformationFoundException: No transformation found: class io.ktor.utils.io.ByteChannelNative -> class Mine
with response from https://httpbin.org/get:
status: 200 OK
response headers:
at kfun:kotlin.Throwable#<init>(){} (0x306ebe)
at kfun:kotlin.Exception#<init>(){} (0x3007b7)
at kfun:kotlin.RuntimeException#<init>(){} (0x300377)
at kfun:kotlin.UnsupportedOperationException#<init>(){} (0x300bf7)
at kfun:io.ktor.client.call.NoTransformationFoundException#<init>(io.ktor.client.statement.HttpResponse;kotlin.reflect.KClass<*>;kotlin.reflect.KClass<*>){} (0x579a53)
at kfun:io.ktor.client.call.HttpClientCall.$receiveCOROUTINE$12#invokeSuspend(kotlin.Result<kotlin.Any?>){}kotlin.Any? (0x578b21)
at kfun:io.ktor.client.call.HttpClientCall#receive(io.ktor.client.call.TypeInfo){}kotlin.Any (0x578fc6)
at kfun:$main$lambda-2COROUTINE$0.invokeSuspend#internal (0x640c1f)
at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} (0x322d4f)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x527128)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x526c9f)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x528c6c)
at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} (0x323007)
Unfinished workers detected, 1 workers leaked!
Use `Platform.isMemoryLeakCheckerActive = false` to avoid this check.
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x527128)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x526c9f)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x528c6c)
at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} (0x323007)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x527128)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x526c9f)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x528c6c)
at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} (0x323007)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x527128)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x526c9f)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x528c6c)
at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} (0x323007)
at kfun:kotlinx.coroutines.DispatchedTask#run(){} (0x4114a6)
at kfun:kotlinx.coroutines.EventLoopImplBase#processNextEvent(){}kotlin.Long (0x3e9ace)
at kfun:kotlinx.coroutines#runEventLoop(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>){} (0x423639)
at kfun:kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal (0x422de0)
at kfun:kotlinx.coroutines#runBlocking(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,0:0>){0§<kotlin.Any?>}0:0 (0x42223b)
at kfun:kotlinx.coroutines#runBlocking$default(kotlin.coroutines.CoroutineContext?;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,0:0>;kotlin.Int){0§<kotlin.Any?>}0:0 (0x422741)
at kfun:#main(){} (0x63e8cc)
at Konan_start (0x641839)
at Init_and_run_start (0x66cdcb)
at __libc_start_main (0x7f0318cb20b3)
at (0x2b8029)
at ((nil))
Here is the related Github issue.
IncorrectDereferenceException thrown when ktor request call from other thread
Hello there, I see that this issue KTOR-499(https://youtrack.jetbrains.com/issue/KTOR-499) is mentioned in Ktor 1.4.1 Change log as fixed (https://github.com/ktorio/ktor/blob/master/CHANGELOG.md) and in the issue in the comments there is a statement that it is working now with the Ktor 1.4.0. I’ve forked the repository(https://github.com/RudolfHladik/ktor-mt-app), updated it to Ktor 1.4.1, Kotlin 1.4.10, Coroutines 1.3.9-native-mt and I get the IncorectDereferenceException . Is there something wrong I do or has the issue reapeared in Ktor 1.4.1?
Core
CIO client engine randomly runs into infinite loop under high load
Stuck in while (!closed)
in io.ktor.network.selector.ActorSelectorManager.process()
.
Probably related to KTOR-717.
Likely a duplicate of KTOR-531.
I also randomly get this:
Exception in thread "main" java.net.BindException: Can't assign requested address
at java.base/sun.nio.ch.Net.connect0(Native Method)
at java.base/sun.nio.ch.Net.connect(Net.java:503)
at java.base/sun.nio.ch.Net.connect(Net.java:492)
at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:751)
at io.ktor.network.sockets.SocketImpl.connect$ktor_network(SocketImpl.kt:31)
at io.ktor.network.sockets.ConnectUtilsJvmKt.connect(ConnectUtilsJvm.kt:19)
at io.ktor.network.sockets.TcpSocketBuilder.connect(TcpSocketBuilder.kt:38)
at io.ktor.client.engine.cio.ConnectionFactory.connect(ConnectionFactory.kt:24)
at io.ktor.client.engine.cio.Endpoint$connect$$inlined$repeat$lambda$1.invokeSuspend(Endpoint.kt:159)
at io.ktor.client.engine.cio.Endpoint$connect$$inlined$repeat$lambda$1.invoke(Endpoint.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout(Undispatched.kt:102)
at kotlinx.coroutines.TimeoutKt.setupTimeout(Timeout.kt:120)
at kotlinx.coroutines.TimeoutKt.withTimeoutOrNull(Timeout.kt:83)
at io.ktor.client.engine.cio.Endpoint.connect(Endpoint.kt:167)
at io.ktor.client.engine.cio.Endpoint$makeDedicatedRequest$1.invokeSuspend(Endpoint.kt:95)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Suppressed: java.net.BindException: Can't assign requested address
... 21 more
Repro:
import io.ktor.application.*
import io.ktor.client.*
import io.ktor.client.engine.cio.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import io.ktor.http.*
import io.ktor.response.*
import io.ktor.routing.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import kotlin.random.*
import kotlin.time.*
import kotlinx.coroutines.*
@OptIn(ExperimentalTime::class)
suspend fun main() = withContext(Dispatchers.Default) {
startServer()
delay(1_000)
val batchSize = 100
val threadCount = 1
HttpClient(CIO) {
engine {
threadsCount = threadCount
}
}.use { client ->
repeat(Int.MAX_VALUE) { i ->
val time = measureTime {
coroutineScope {
repeat(batchSize) {
launch {
client.get<HttpResponse>("http://localhost:8080").readText()
}
}
}
}
println("${i + 1}: $time")
}
}
}
private fun startServer() {
embeddedServer(Netty, port = 8080) {
val random = Random(0)
val data = buildString(100_000) { repeat(100_000) { append(random.nextInt(0, 10)) } }
routing {
get("/") {
call.respondText(data, ContentType.Text.Plain)
}
}
}.start()
}
Infrastructure
ktor-server-test-host indirectly depends on logback-classic
Using ktor-server-test-host
results in logback classic being pulled in as a test dependancy, the relevent dependancy tree is below.
+--- io.ktor:ktor-server-test-host -> 1.3.2
| +--- io.ktor:ktor-client-tests-jvm:1.3.2
| | +--- ch.qos.logback:logback-classic:1.2.3
| | | +--- ch.qos.logback:logback-core:1.2.3
| | | \--- org.slf4j:slf4j-api:1.7.25 -> 1.7.30
Add artifact dependency to release configuration
IntelliJ IDEA Plugin
Plugin: "Could not find io.ktor:ktor-client-http-timeout" when building project created with http-timeout feature
Steps:
- Install Ktor Plugin
- Create Project With the Wizzard, like in the screenshot
- Build/Run
Expectation: Application should build an run:
Build does not work, because the ktor-client-http-timeout dependency can not be found.
I checked it on the server: https://kotlin.bintray.com/ktor/io/ktor/. I can not find this dependency there.
Plugin: the Ktor project type tooltip in the New Project wizard contains text used for Java projects
Steps:
- Open IntelliJ IDEA 2020.3 EAP (IU-203.3645.34)
- Click New Project on the Welcome screen.
- Hover the mouse pointer over Ktor.
ACT: A tooltip contains the text for a Java project.
EXP: A tooltip contains Ktor-specific text, for example: 'Create a Ktor project with a specific set of features'.
Test compilation errors when project created via plugin with "Mock HttpClient Engine"
folders as created by the ktor idea plugin:
/
.gradle/
resources/
src/
test/
expected folders
/
.gradle/
src/
main/
kotlin/
java/ -- if i would have java src
resources/
test/
kotlin/
java/ -- if i would have java src
resources/
More details here: https://docs.gradle.org/current/userguide/organizing_gradle_projects.html#sec:separate_language_source_files
There's a secondary issue with the generated sources: the test file has compilation errors.
I've attached multiple pictures, along with the generated project.
No changelog for 1.4.0 and 1.4.1 on the plugin page in IDE Marketplace
Server
Sensible defaults for compression feature
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/544
At the moment Ktor is trying to compress even the steel.
I added this method in my core lib to overcome that.
I think there should be a similar thing active by default. What do you reckon?
fun Application.installDefaultCompresion() {
install(Compression) {
gzip {
matchContentType(
ContentType.Text.Any,
ContentType.Application.JavaScript,
ContentType.Application.Json,
ContentType.Application.Rss,
ContentType.Application.Xml,
ContentType.Application.Xml_Dtd,
ContentType.Application.Atom,
ContentType.Image.SVG,
ContentType.Image.XIcon
)
minimumSize(1400) // not worth compressing
}
}
}
Setting multiple cache control directives is impossible with current API
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/557
Using the install for CachingHeaders I want to configure it in such a way it produces this header:
Cache-Control: no-cache, no-store, must-revalidate
I cannot do it.
For example I tried this:
options { outgoingContent -> when (outgoingContent.contentType?.withoutParameters()) { ContentType.Text.CSS -> CachingOptions(CacheControl.NoCache(null)) else -> null } } options { outgoingContent -> when (outgoingContent.contentType?.withoutParameters()) { ContentType.Text.CSS -> CachingOptions(CacheControl.NoStore(null)) else -> null } }
it makes it twice:
cache-control: no-cache
cache-control: no-store
The best would be if options can return a lamda with a list of CachingOptions as return value, then the system would add all of them. Or allow caching options to accept varargs?
Also, why is "must revalidate" tied to max age?
See this, it can be also on it's own:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#Examples
Netty: NumberFormatException when make a request via HTTP/2 without explicit port
Hello.
There is a bug with Http2LocalConnectionPoint. If there is no port specified in authority then it tries to parse domain as port.
I've created a project to reproduce the bug: https://github.com/Vlad8161/netty-bug-sample
To reproduce it run the server on 443 port and access it without explicit port specifying.
Stack Trace:
java.lang.NumberFormatException: For input string: "localhost"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.parseInt(Integer.java:615)
at io.ktor.server.netty.http2.Http2LocalConnectionPoint.getPort(Http2LocalConnectionPoint.kt:31)
at io.ktor.features.HSTS.intercept(HSTS.kt:87)
at io.ktor.features.HSTS$Feature$install$1.invokeSuspend(HSTS.kt:101)
at io.ktor.features.HSTS$Feature$install$1.invoke(HSTS.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
at io.ktor.features.StatusPages$interceptCall$2.invokeSuspend(StatusPages.kt:101)
at io.ktor.features.StatusPages$interceptCall$2.invoke(StatusPages.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:91)
at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:189)
at io.ktor.features.StatusPages.interceptCall(StatusPages.kt:100)
at io.ktor.features.StatusPages$Feature$install$2.invokeSuspend(StatusPages.kt:140)
at io.ktor.features.StatusPages$Feature$install$2.invoke(StatusPages.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:139)
at io.ktor.features.CallLogging$Feature$install$2.invoke(CallLogging.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:121)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:243)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:113)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:133)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:77)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:54)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.startCoroutineImpl(Builders.common.kt:182)
at kotlinx.coroutines.BuildersKt.startCoroutineImpl(Unknown Source)
at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:145)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:54)
at kotlinx.coroutines.BuildersKt.launch(Unknown Source)
at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:37)
at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:29)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
The problem is at Http2LocalConnectionPoint:31
I think is would be better to do something like this:
nettyHeaders.authority()?.toString()?.substringAfter(":", if (scheme == "https") "443" else "80")?.toInt()
because often local port information is not enough
Thanks in advance :)
Class cast exception on web-socket cancelation
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1929
Ktor Version and Engine Used (client or server and name)
ktor-cio server 1.3.2
Describe the bug
I am getting an internal error when force-closing server with opened web-socket connection
Here is the stack trace:
13:32:20.839 [DefaultDispatcher-worker-4] ERROR io.ktor.server.cio.HttpServer - Unhandled exception caught for CoroutineName(http-pipeline-writer)
kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[ExperimentalCoroutineDispatcher@45e6d6a4[scheduler = DefaultDispatcher@6341301e[Pool Size {core = 10, max = 1024}, Worker States {CPU = 6, blocking = 0, parked = 8, dormant = 0, terminated = 0}, running workers queues = [0c, 0c, 0c, 0c, 0c, 0c], global CPU queue size = 5, global blocking queue size = 0, Control State {created workers= 14, blocking tasks = 0, CPUs acquired = 6}]], Continuation at io.ktor.server.cio.backend.ServerPipelineKt$startServerConnectionPipeline$1$outputsActor$1.invokeSuspend(ServerPipeline.kt:49)@2482cf33]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core(DispatchedTask.kt:93)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:64)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Caused by: java.lang.ClassCastException: class kotlin.coroutines.jvm.internal.CompletedContinuation cannot be cast to class kotlinx.coroutines.DispatchedContinuation (kotlin.coroutines.jvm.internal.CompletedContinuation and kotlinx.coroutines.DispatchedContinuation are in unnamed module of loader 'app')
at kotlinx.coroutines.CoroutineDispatcher.releaseInterceptedContinuation(CoroutineDispatcher.kt:103)
at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted(ContinuationImpl.kt:118)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:39)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:55)
... 4 common frames omitted
CORS - request with non-simple content type not failing
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1906
Ktor Version and Engine Used
Ktor 1.3.2, Netty
Here is the failing test:
@Test
fun testRequestWithNonSimpleContentTypeShouldFail() {
withTestApplication {
application.install(CORS) {
anyHost()
}
application.routing {
post("/") {
call.respond(HttpStatusCode.OK)
}
}
handleRequest(HttpMethod.Post, "/") {
addHeader(HttpHeaders.Origin, "http://my-host")
addHeader(HttpHeaders.ContentType, "application/json")
setBody("""{"cors":"test"}""")
}.apply {
assertEquals(HttpStatusCode.Forbidden, response.status())
}
}
}
I've read it that CORS feature doesn’t allow non-simple request body content types by default but this test fails. Am I understanding it wrong?
Scheme instead of version in OriginConnectionPoint
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1869
Ktor Version and Engine Used (client or server and name)
1.3.2, server
Describe the bug
ApplicationCall.request.origin.remoteHost
holds the scheme instead of the version (like 'http' instead of HTTP/1.1
).
The root cause is this line.
I provided a fix here: https://github.com/ktorio/ktor/pull/1871
To Reproduce
Steps to reproduce the behavior:
- Configure request logging for a ktor server
- Install 'XForwardedHeaderSupport`
- print
call.request.origin.remoteHost
- Run the server behind a proxy
- Send a request to the proxy
- Observe that
http
is printed
Expected behavior
HTTP/1.1
should be printed
Make Jackson serialization stream response
From user's PR:
The existing implementation of the JacksonConverter used a TextContent,
which meant that the ObjectMapper had to serialize the entire output
value to a String before responding.For large output objects, this causes significant allocation preassure.
It can avoid by instead streaming the output.
Test Infrastructure
Replace JUnit with kotlin.test
Replace JUnit with kotlin.test as much as possible
Reason
- Better Kotlin support, like assertEquals
- theoretical platform independent
Other
Create sample with native CIO client and `native-mt` coroutines
Fix HTTP2 tests with new versions of java8
Blocking calls (ReentrantLock.lock) on Apache engine dispatcher threads
I'm using BlockHound to check if there are any blocking calls within my coroutines that aren't using IO dispatcher.
BlockHound reports the following occasion for Ktor client with Apache engine on thread ktor-apache-dispatcher-worker-4
.
Maybe it's a bug, maybe it's a misconfiguration of the dispatcher threads as CPU-bound.
reactor.blockhound.BlockingOperationError: Blocking call! jdk.internal.misc.Unsafe#park
at java.base/jdk.internal.misc.Unsafe.park(Unsafe.java)
at java.base/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:917)
at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1240)
at java.base/java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:267)
at org.apache.http.nio.pool.AbstractNIOConnPool.lease(AbstractNIOConnPool.java:278)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.requestConnection(PoolingNHttpClientConnectionManager.java:295)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.requestConnection(AbstractClientExchangeHandler.java:377)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.start(DefaultClientExchangeHandlerImpl.java:129)
at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:141)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:68)
at io.ktor.client.engine.apache.ApacheHttpRequestKt.sendRequest(ApacheHttpRequest.kt:33)
at io.ktor.client.engine.apache.ApacheEngine.execute(ApacheEngine.kt:37)
at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:86)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Url(…) doesn't parse non-HTTP-like URL strings correctly
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1204
Url
is supposed to not just support HTTP-like URLs but any kind of URLs.
However the string parser seems to properly support only HTTP-like URLs:
Example
Url("mailto:marc@knaup.io").toString()
Url("file:///var/www").toString()
Expected
"mailto:marc@knaup.io"
"file:///var/www"
Actual
"mailto://localhost/marc@knaup.io"
"file://var/www"
Ktor 1.2.2
"response.readText()" never returns (probably infinite loop)
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/768
Ktor Version
1.0.0
Ktor Engine Used(client or server and name)
Apache
JVM Version, Operating System and Relevant Context
JDK 1.8.x, Win / Linux
Feedback
After about 1-2 hours of constant requests using ktor client it unexpectedly hangs on line:
val data = response.readText()
ktor will never pass this line. Even after 9 hours it's still on this line.
No exceptions are thrown and the process is not closed.
This might do something with the response from the server because it's unpredictable at all. It might happen after 10 minutes and might be after 6 hours.
Are there any suggestions what this can be or how can I investigate this issue?
Maybe, there are some timeouts that I need to set?
Thank you.
Ktor CIO client, connect timeout handling
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1720
- Using the following code (full example at https://github.com/rrva/ktor-client-connection-leak)
- Run the main method repeatedly. Sometimes, all futures created will not complete . Why? Expected behavior is that all created futures will complete eventually, so
Done with all
will be printed.
class Client(val fooServiceUrl: String) {
val client = HttpClient(CIO) {
engine {
requestTimeout = 500
endpoint.connectTimeout = 100
endpoint.connectRetryAttempts = 2
}
install(JsonFeature) {
serializer = KotlinxSerializer(
Json(
JsonConfiguration(
isLenient = true,
ignoreUnknownKeys = true,
serializeSpecialFloatingPointValues = true,
useArrayPolymorphism = true
)
)
)
}
}
suspend fun fetchFoo(ids: List<String>): List<Foo> {
return client.request {
header("User-Agent", "Foo")
parameter("input", ids.joinToString(","))
url("$fooServiceUrl/foo")
}
}
}
@Serializable
data class Foo(val id: String)
val client = Client("http://localhost:9090")
fun fetchFooAsync(ids:List<String>): CompletableFuture<List<Foo>> {
return CoroutineScope(Job() + Dispatchers.IO + MDCContext()).async {
try {
client.fetchFoo(ids)
} catch (e: Throwable) {
println(e.message)
listOf<Foo>(Foo("timeout"))
}
}.asCompletableFuture()
}
fun main() {
HttpServer.create(InetSocketAddress(9090), 1000).apply {
createContext("/foo") { http ->
http.responseHeaders.add("Content-type", "application/json")
http.sendResponseHeaders(200, 0)
PrintWriter(http.responseBody).use { out ->
out.println("""[{"id":"1"}]""")
}
}
start()
}
println("Start")
val requests = (1..100).map {
fetchFooAsync(listOf(it.toString()))
}
for (request in requests) {
println("Waiting for coroutine...")
println(request.get())
println("Done waiting")
}
println("Done with all")
}
Reproduced on 1.3.1
Extract flaky tests from `master` to the `flaky-tests` branch
We need this to improve general stability and measure how many flaky tests we have
Split ktor-server-test-host module
Ktor Client Null Pointer Exception (HTTPS)
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1890
Ktor Version and Engine Used (client or server and name)
1.3.1 (client)
Describe the bug
When sending a POST request over HTTPS, the ktor client will intermittently throw a NPE.
To Reproduce
Not sure exactly how to reproduce... given that the failure is intermittent. I get this NPE approximately 50% of the time.
Screenshots
kotlin.KotlinNullPointerException
at io.ktor.network.tls.TLSClientHandshake.handleServerDone(TLSClientHandshake.kt:327)
at io.ktor.network.tls.TLSClientHandshake.handleCertificatesAndKeys(TLSClientHandshake.kt:291)
at io.ktor.network.tls.TLSClientHandshake.negotiate(TLSClientHandshake.kt:155)
at io.ktor.network.tls.TLSClientHandshake$negotiate$1.invokeSuspend(TLSClientHandshake.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:561)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:727)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:667)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:655)
TestHttpClientEngine KNPE
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1891
Ktor Version and Engine Used (client or server and name)
1.3.2, test client/server, ktor-server-test-host
Describe the bug
Take this example.
fun main() {
withTestApplication(Application::helloModule) {
runBlocking {
val res = client.get<HttpResponse>("NonExistenceRoute")
println(res)
}
}
}
fun Application.helloModule() {
routing {
get {
call.respond(HttpStatusCode.OK, "Hello")
}
}
}
Throws KNPE because route doesn't exists. If you follow stack trace, it's obvious where KNPE was thrown.
It's pretty nice to have this kind of tests. Is TestHttpClientEngine
in experimental state? I also had another issue where the tests would stuck using multiparts.
Expected behavior
A response with 404 not found status.
Screenshots
Stack trace of the thrown exception,
Exception in thread "main" kotlin.KotlinNullPointerException
at io.ktor.server.testing.client.TestHttpClientEngine.execute(TestHttpClientEngine.kt:37)
at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:83)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
1.4.1
released 24th September 2020
Client
"FreezingException: freezing of InvokeOnCompletion has failed" using native-mt coroutines
Hi.
I have a problem on Ktor client native
.
I’ve been struggling to make the code below works after migrating kotlin to 1.4.0
version
while migrating, I changed coroutine version to 1.3.9-native-mt
class TestFreezingViewModel(val client: HttpClient) {
fun call() {
CoroutineScope(Dispatchers.Main).launch(start = CoroutineStart.LAZY) {
try {
client.get<String>("https://test.com") {}
} catch (e: Throwable) {
e.printStackTrace()
}
}.start()
}
}
swift side code
TestFreezingViewModel().call()
while the code running, The error below occurs
kotlin.native.concurrent.FreezingException: freezing of InvokeOnCompletion[InvokeOnCompletion@1b76e08] has failed, first blocker is HttpClient[io.ktor.client.engine.ios.IosClientEngine@1b1b748]
at 0 kotlinIOS 0x000000010bf93fed kfun:kotlin.Throwable#<init>(kotlin.String?){} + 93
at 1 kotlinIOS 0x000000010bf8cb9b kfun:kotlin.Exception#<init>(kotlin.String?){} + 91
at 2 kotlinIOS 0x000000010bf8cdeb kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 91
at 3 kotlinIOS 0x000000010bfc2c13 kfun:kotlin.native.concurrent.FreezingException#<init>(kotlin.Any;kotlin.Any){} + 643
at 4 kotlinIOS 0x000000010bfc4297 ThrowFreezingException + 231
at 5 kotlinIOS 0x000000010c0b2f4f FreezeSubgraph + 2815
at 6 kotlinIOS 0x000000010c0cc17b Kotlin_Worker_freezeInternal + 27
at 7 kotlinIOS 0x000000010bfc2d3e kfun:kotlin.native.concurrent#freeze@0:0(){0§<kotlin.Any?>}0:0 + 62
at 8 kotlinIOS 0x000000010c10c749 kfun:kotlinx.coroutines.JobSupport#invokeOnCompletion(kotlin.Boolean;kotlin.Boolean;kotlin.Function1<kotlin.Throwable?,kotlin.Unit>){}kotlinx.coroutines.DisposableHandle + 1881
at 9 kotlinIOS 0x000000010c10bf76 kfun:kotlinx.coroutines.JobSupport#invokeOnCompletion(kotlin.Function1<kotlin.Throwable?,kotlin.Unit>){}kotlinx.coroutines.DisposableHandle + 198
at 10 kotlinIOS 0x000000010c33b15b kfun:io.ktor.client.engine#createCallContext@io.ktor.client.engine.HttpClientEngine(kotlinx.coroutines.Job){}kotlin.coroutines.CoroutineContext + 1451
at 11 kotlinIOS 0x000000010c302d08 kfun:io.ktor.client.engine.HttpClientEngine.$executeWithinCallContextCOROUTINE$9.invokeSuspend#internal + 1048
at 12 kotlinIOS 0x000000010c303444 kfun:io.ktor.client.engine.HttpClientEngine.executeWithinCallContext#internal + 308
at 13 kotlinIOS 0x000000010c30420a kfun:io.ktor.client.engine.HttpClientEngine.$install$lambda-0COROUTINE$10.invokeSuspend#internal + 1562
at 14 kotlinIOS 0x000000010c304bf8 kfun:io.ktor.client.engine.HttpClientEngine.$install$lambda-0COROUTINE$10.invoke#internal + 312
at 15 kotlinIOS 0x000000010c29da5f kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1135
at 16 kotlinIOS 0x000000010c29d0ab kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal + 395
at 17 kotlinIOS 0x000000010c29d50a kfun:io.ktor.util.pipeline.SuspendFunctionGun.execute#internal + 474
at 18 kotlinIOS 0x000000010c295759 kfun:io.ktor.util.pipeline.Pipeline#execute(1:1;1:0){}1:0 + 361
at 19 kotlinIOS 0x000000010c326be5 kfun:io.ktor.client.features.HttpSend.DefaultSender.$executeCOROUTINE$27.invokeSuspend#internal + 1365
at 20 kotlinIOS 0x000000010c327244 kfun:io.ktor.client.features.HttpSend.DefaultSender.execute#internal + 308
at 21 kotlinIOS 0x000000010c324fbc kfun:io.ktor.client.features.HttpSend.Feature.$install$lambda-0COROUTINE$26.invokeSuspend#internal + 2236
at 22 kotlinIOS 0x000000010c3260f8 kfun:io.ktor.client.features.HttpSend.Feature.$install$lambda-0COROUTINE$26.invoke#internal + 312
at 23 kotlinIOS 0x000000010c29da5f kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1135
at 24 kotlinIOS 0x000000010c29d0ab kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal + 395
at 25 kotlinIOS 0x000000010c29d2a5 kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceedWith#internal + 213
at 26 kotlinIOS 0x000000010c3173eb kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-0COROUTINE$17.invokeSuspend#internal + 907
at 27 kotlinIOS 0x000000010c317c58 kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-0COROUTINE$17.invoke#internal + 312
at 28 kotlinIOS 0x000000010c29da5f kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal + 1135
at 29 kotlinIOS 0x000000010c29d0ab kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal + 395
when Coroutine’s invokeOnCompletion
is invoked, freeze()
is called.
and it tells that HttpClient
can’t be frozen
When I checked the source code
HttpClient.init
calls preventFreeze()
I would like to know the intention of preventFreeze
and what is the proper approach to use on developer’s side.
please check if this is bug, other wise, kindly let me share some advice.
Thanks
CIO: java.io.EOFException when server requires client certificate and restricts the authority
CIO tries to read more bytes as available in the packet, see pull request https://github.com/ktorio/ktor/pull/2063.
This results into a crash, and no retry is possible, because false handling of the authoritiesSize.
java.io.EOFException: Premature end of stream: expected 2 bytes
Example:
reading | remainingBefore | remainingAfter | content | position | |
---|---|---|---|---|---|
before | 281 | 281 | |||
reading authorizationSize | 2 | 281 | 279 | 279 | |
reading size | 2 | 279 | 277 | 91 | |
reading CA 1 | 91 | 277 | 186 | CA... | 91 (should 93) |
reading size | 2 | 186 | 184 | 91 | |
reading CA 2 | 91 | 184 | 93 | CA | 182 (should 186) |
reading size | 2 | 93 | 91 | 91 | |
reading CA 3 | 91 | 91 | 0 | CA | 273 (should 279) |
Calling HttpStatement#toString more than once throws IllegalArgumentException
Ktor v1.4.0
This can be trivially reproduced with the following code:
fun main() {
// Using OkHttp for illustrative purposes; it doesn't matter what engine
val client = HttpClient(OkHttp)
val statement = HttpStatement(HttpRequestBuilder(), client)
println(statement.toString())
println(statement.toString())
}
Expected output:
HttpStatement[http://localhost/]
HttpStatement[http://localhost/]
Actual output:
HttpStatement[http://localhost/]
Exception in thread "main" java.lang.IllegalArgumentException: ParametersBuilder can only build a single Parameters instance
at io.ktor.http.ParametersBuilder.build(Parameters.kt:32)
at io.ktor.http.URLBuilder.appendTo(URLBuilder.kt:67)
at io.ktor.http.URLBuilder.buildString(URLBuilder.kt:81)
at io.ktor.client.statement.HttpStatement.toString(HttpStatement.kt:138)
at MainKt.main(Main.kt:11)
at MainKt.main(Main.kt)
Process finished with exit code 1
This is because of:
HttpStatement#toString
callsURLBuilder#buildString
, which callsURLBuilder#appendTo
URLBuilder#appendTo
callsParametersBuilder#build
ParametersBuilder#build
can only be called once, per this assertion:
require(!built) { "ParametersBuilder can only build a single Parameters instance" }
Implement runtime check of using `native-mt` coroutines
Ktor websocket client passes configured max frame as timeout millis
The client passes maxFrameSize instead of timeoutMillis as the third argument.
1.4.0: breaking change by making response nullable in ResponseException
Link to breaking change: https://github.com/ktorio/ktor/commit/9785a7a82e89881ad279a77ac6adedfe621fb0b8#r41593709
Either this change should have been kept for a future version, or this version should be labeled 2.0.0
Parser Exception in header with character code 1 not allowed
The following URL
https://data.smartdublin.ie/cgi-bin/rtpi
returns valid results with other HTTP clients, including IntelliJ HTTP Requests. However with Ktor, independently of engine, it throws a parser exception (side note, it would be good to indicate the header in the exception result)
Exception in thread "main" io.ktor.http.cio.ParserException: Character with code 1 is not allowed in header names,
HTTP/1.1 200 OKServer: nginx/1.17.8Date: Sun, 19 Jul 2020 07:12:42 GMTContent-Type: application/json; charset=utf-8Content-Length: 1596054Connection: keep-aliveX-Powered-By: ASP.NETSet-Cookie: visid_incap_1822411=6ycr4UzsSv+6sdm+QnNmpmnyE18AAAAAQUIPAAAAAACm9JYEnYu4a0fTA7VxKyxu; expires=Sun, 18 Jul 2021 08:40:35 GMT; HttpOnly; path=/; Domain=.nationaltransport.ieSet-Cookie: incap_ses_535_1822411=vjDoPwoqvTdguklauLNsB2nyE18AAAAAoieq1zxaD46bWhzEJyQd3w==; path=/; Domain=.nationaltransport.ieSet-Cookie: ___utmvmzauvysSB=WQYQExBjiZv; path=/; Max-Age=900Set-Cookie: ___utmvazauvysSB=TjhnJRz; path=/; Max-Age=900
at io.ktor.http.cio.HttpParserKt.characterIsNotAllowed(HttpParser.kt:279)
at io.ktor.http.cio.HttpParserKt.parseHeaderValue(HttpParser.kt:261)
at io.ktor.http.cio.HttpParserKt.parseHeaders(HttpParser.kt:119)
at io.ktor.http.cio.HttpParserKt.parseResponse(HttpParser.kt:73)
at io.ktor.http.cio.HttpParserKt$parseResponse$1.invokeSuspend(HttpParser.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Process finished with exit code 1
To reproduce see attached code.
Native: InvalidMutabilityException creating HttpClient
As of v.1.4.0, ktor client is not working on Kotlin/native iOS due to an InvalidMutabilityException.
I tested with both coroutines "1.3.9" as well as "1.3.9-native-mt", each produce the same error:
Uncaught Kotlin exception: kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen kotlinx.coroutines.ChildHandleNode@3873b88
at 0 jgosdk 0x000000010b7376fd kfun:kotlin.Throwable#<init>(kotlin.String?){} + 93 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Throwable.kt:23:37)
at 1 jgosdk 0x000000010b73030b kfun:kotlin.Exception#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
at 2 jgosdk 0x000000010b73055b kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
at 3 jgosdk 0x000000010b767deb kfun:kotlin.native.concurrent.InvalidMutabilityException#<init>(kotlin.String){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/concurrent/Freezing.kt:22:60)
at 4 jgosdk 0x000000010b7682c2 ThrowInvalidMutabilityException + 690 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/concurrent/Internal.kt:92:11)
at 5 jgosdk 0x000000010b85e9ec MutationCheck + 108
at 6 jgosdk 0x000000010b9017b6 kfun:kotlinx.coroutines.internal.LinkedListNode#<set-_next>(kotlinx.coroutines.internal.LinkedListNode){} + 102 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/internal/LinkedList.kt:19:28)
at 7 jgosdk 0x000000010b901d9b kfun:kotlinx.coroutines.internal.LinkedListNode#addLast(kotlinx.coroutines.internal.LinkedListNode){} + 283 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/internal/LinkedList.kt:31:9)
at 8 jgosdk 0x000000010b9020e6 kfun:kotlinx.coroutines.internal.LinkedListNode#addOneIfEmpty(kotlinx.coroutines.internal.LinkedListNode){}kotlin.Boolean + 230 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/internal/LinkedList.kt:47:9)
at 9 jgosdk 0x000000010b8a3cbf kfun:kotlinx.coroutines.JobSupport.promoteSingleToNodeList#internal + 463 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:532:15)
at 10 jgosdk 0x000000010b8a291d kfun:kotlinx.coroutines.JobSupport#invokeOnCompletion(kotlin.Boolean;kotlin.Boolean;kotlin.Function1<kotlin.Throwable?,kotlin.Unit>){}kotlinx.coroutines.DisposableHandle + 2061 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:470:25)
at 11 jgosdk 0x000000010b8a208f kfun:kotlinx.coroutines.JobSupport#invokeOnCompletion(kotlin.Function1<kotlin.Throwable?,kotlin.Unit>){}kotlinx.coroutines.DisposableHandle + 207 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:449:9)
at 12 jgosdk 0x000000010ba59d04 kfun:io.ktor.client#HttpClient(io.ktor.client.engine.HttpClientEngineFactory<0:0>;kotlin.Function1<io.ktor.client.HttpClientConfig<0:0>,kotlin.Unit>){0§<io.ktor.client.engine.HttpClientEngineConfig>}io.ktor.client.HttpClient + 1156 (/opt/buildAgent/work/a85294440dc5c6e/ktor-client/ktor-client-core/common/src/io/ktor/client/HttpClient.kt:45:36)
Core
Upgrade kotlinx.serialization to 1.0.0-RC2
ConcurrentList.increaseCapacity() throws ArrayIndexOutOfBoundsException
I have a test that uses MockEngine which makes more than 32 calls, and then throws a java.lang.ArrayIndexOutOfBoundsException (stacktrace at the bottom).
I tracked the problem down to io.ktor.util.collections.ConcurrentList.increaseCapacity()
(ConcurrentList.kt:180):
private fun increaseCapacity(targetCapacity: Int = data.size * 2) {
val newData = SharedList<T>(targetCapacity)
for (index in 0 until targetCapacity) {
newData[index] = data[index]
}
data = newData
}
It is clear from this code that the for loop uses targetCapacity
which is going to try to access data outside of the original data size. Here is what the code should be:
private fun increaseCapacity(targetCapacity: Int = data.size * 2) {
val newData = SharedList<T>(targetCapacity)
for (index in 0 until data.size) {
newData[index] = data[index]
}
data = newData
}
Here is the stacktrace for reference.
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 32 out of bounds for length 32
at java.base/java.lang.invoke.VarHandle$1.apply(VarHandle.java:2011)
at java.base/java.lang.invoke.VarHandle$1.apply(VarHandle.java:2008)
at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:159)
at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:156)
at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:62)
at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
at java.base/java.lang.invoke.VarHandleObjects$Array.getVolatile(VarHandleObjects.java:438)
at java.base/java.lang.invoke.VarHandleGuards.guard_LI_L(VarHandleGuards.java:646)
at java.base/java.util.concurrent.atomic.AtomicReferenceArray.get(AtomicReferenceArray.java:100)
at io.ktor.util.collections.internal.SharedList.get(SharedList.kt:35)
at io.ktor.util.collections.ConcurrentList.increaseCapacity(ConcurrentList.kt:180)
at io.ktor.util.collections.ConcurrentList.increaseCapacity$default(ConcurrentList.kt:177)
at io.ktor.util.collections.ConcurrentList.add(ConcurrentList.kt:65)
at io.ktor.client.engine.mock.MockEngine.execute(MockEngine.kt:63)
at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:86)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedContinuationKt.resumeCancellableWith(DispatchedContinuation.kt:333)
at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:26)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.startCoroutineImpl(Builders.common.kt:188)
at kotlinx.coroutines.BuildersKt.startCoroutineImpl(Unknown Source)
at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:145)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.async(Builders.common.kt:91)
at kotlinx.coroutines.BuildersKt.async(Unknown Source)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.async$default(Builders.common.kt:84)
at kotlinx.coroutines.BuildersKt.async$default(Unknown Source)
at io.ktor.client.engine.HttpClientEngine$DefaultImpls.executeWithinCallContext(HttpClientEngine.kt:81)
at io.ktor.client.engine.HttpClientEngine$install$1.invokeSuspend(HttpClientEngine.kt:66)
at io.ktor.client.engine.HttpClientEngine$install$1.invoke(HttpClientEngine.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:323)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:168)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:188)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:31)
at io.ktor.client.features.HttpSend$DefaultSender.execute(HttpSend.kt:124)
at io.ktor.client.features.HttpSend$Feature$install$1.invokeSuspend(HttpSend.kt:88)
at io.ktor.client.features.HttpSend$Feature$install$1.invoke(HttpSend.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:323)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:168)
at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(PipelineContext.kt:178)
at io.ktor.client.features.HttpCallValidator$Companion$install$1.invokeSuspend(HttpCallValidator.kt:86)
at io.ktor.client.features.HttpCallValidator$Companion$install$1.invoke(HttpCallValidator.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:323)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:168)
at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invokeSuspend(HttpRequestLifecycle.kt:37)
at io.ktor.client.features.HttpRequestLifecycle$Feature$install$1.invoke(HttpRequestLifecycle.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:323)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:168)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:188)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:31)
at io.ktor.client.HttpClient.execute(HttpClient.kt:185)
at io.ktor.client.statement.HttpStatement.executeUnsafe(HttpStatement.kt:104)
at com.gatsby.trade.client.OrbisOrderClient$placeOrder$$inlined$post$1.invokeSuspend(OrbisClient.kt:242)
Infrastructure
Enable binary-compatibility-validator in all modules
Samples
Routing: get matcher has higher priority than param matcher on the same level
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/378
Feel free to rename this issue, once the problem is located. Reported by @qweryty at slack:
Giving /test?error=error
and this snippet, the param
route is not handled:
fun main(args: Array<String>) {
val server = embeddedServer(Netty, 9995) {
install(DefaultHeaders)
install(Compression)
install(CallLogging)
routing {
route("test") {
param("error") {
handle {
call.respondText { "error" } // Not handled
}
}
get {
call.respondText { "test" }
}
}
}
}
server.start()
}
With the trace feature from the wip 0.9.2, this is the output:
// /, segment:0 -> SUCCESS @ /test/(method:GET))
// /test, segment:1 -> SUCCESS @ /test/(method:GET))
// /test/[error], segment:1 -> SUCCESS @ /test/[error])
// /test/(method:GET), segment:1 -> SUCCESS @ /test/(method:GET))
Even when not the same, since the previous sample should match all the methods for the param, this other sample would work (matching only get requests for the error param):
routing {
route("test") {
method(HttpMethod.Get) {
param("error") {
handle {
call.respondText { "error" } // Matched
}
}
handle {
call.respondText { "test" }
}
}
}
}
// /, segment:0 -> SUCCESS; Parameters [error=[error]] @ /test/(method:GET)/[error])
// /test, segment:1 -> SUCCESS; Parameters [error=[error]] @ /test/(method:GET)/[error])
// /test/(method:GET), segment:1 -> SUCCESS; Parameters [error=[error]] @ /test/(method:GET)/[error])
// /test/(method:GET)/[error], segment:1 -> SUCCESS @ /test/(method:GET)/[error])
[SECURITY] 0.9.* Releases were built/executed/released in the context of insecure/untrusted code
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/953
CWE-829: Inclusion of Functionality from Untrusted Control Sphere
All of these build files include resolving these dependencies over HTTP instead of HTTPS. Any of these artifacts could have been MITM to maliciously compromise them and infect the build artifacts that were produced. Additionally, if any of these JAR files were compromised, any developers using these dependencies now could continue to be infected.
- https://github.com/ktorio/ktor/blob/0.9.5/build.gradle
- https://github.com/ktorio/ktor/blob/0.9.4/build.gradle
- https://github.com/ktorio/ktor/blob/0.9.3/build.gradle
- https://github.com/ktorio/ktor/blob/0.9.2/build.gradle
- https://github.com/ktorio/ktor/blob/0.9.1/build.gradle
- https://github.com/ktorio/ktor/blob/1.0.0/build.gradle
- https://github.com/ktorio/ktor/blob/1.0.1/build.gradle
This vulnerability has a CVSS v3.0 Base Score of 8.1/10
https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator?vector=AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H
This isn't just theoretical; POC code exists already to maliciously compromise jar file inflight.
See:
Please remove or update these samples, because they do not compile with new versions of Kotlin
This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/74
Especially the MPP sample. Please don't make developer's lives more difficult by letting broken code rot here on GitHub.
Form authorization not working in example (feature/auth)
This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/19
Not only are the example outdated (code is deprecated), but its also not working, thus causing quite a lot of confusion; (especially since the documentation regarding the Form authorization is pretty limited);
I can create a PR for a suitable minor change to make it work if you want :)
Need help compiling fullstack-mpp.
This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/37
I've tried to build fullstack-mpp project by opening build.gradle in IntelliJ Idea and then adding gradle.properties file with following content:
ktor_version=1.1.1
kotlin.code.style=official
kotlin_version=1.3.11
logback_version=1.2.1
When trying to run it, I get following message:
FAILURE: Build failed with an exception.
* Where:
Build file 'C:\Users\czare\IdeaProjects\fullstack-mpp\build.gradle' line: 81
* What went wrong:
Could not determine the dependencies of task ':run'.
> No such property: files for class: org.jetbrains.kotlin.gradle.plugin.mpp.DefaultKotlinCompilationOutput
I didn't change anything in build.gradle file. What am I doing wrong?
Missing dependency io.ktor:ktor-client-ios_debug_ios_arm64:1.1.1
This issue was imported from GitHub issue: https://github.com/ktorio/ktor-samples/issues/35
Using commit 5480a4d8eb338302f181c054b2a067e67ff7e366
- git clone
- ./gradlew build
- Error:
Could not resolve all files for configuration ':client-mpp:iosArm64CompileKlibraries'.
Could not find io.ktor:ktor-client-ios_debug_ios_arm64:1.1.1.
Server
Exception after WebSocketSession.close() invocation.
Invoke WebSocketSession.close()
It causes the following exception:
kotlinx.coroutines.experimental.channels.ClosedSendChannelException: Channel was closed
at kotlinx.coroutines.experimental.channels.Closed.getSendException(AbstractChannel.kt:1008)
at kotlinx.coroutines.experimental.channels.AbstractSendChannel.offer(AbstractChannel.kt:185)
at kotlinx.coroutines.experimental.channels.AbstractSendChannel.send(AbstractChannel.kt:175)
at kotlinx.coroutines.experimental.channels.ChannelCoroutine.send$suspendImpl(ChannelCoroutine.kt:33)
at kotlinx.coroutines.experimental.channels.ChannelCoroutine.send(ChannelCoroutine.kt)
at kotlinx.coroutines.experimental.channels.LazyActorCoroutine.send(Actor.kt:192)
at io.ktor.http.cio.websocket.WebSocketWriter.flush(WebSocketWriter.kt:122)
at io.ktor.http.cio.websocket.RawWebSocket.flush(RawWebSocket.kt:37)
at io.ktor.websocket.DelegatedWebSocketServerSession.flush(WebSocketServerSession.kt)
at io.ktor.http.cio.websocket.DefaultWebSocketSessionImpl.flush(DefaultWebSocketSessionImpl.kt)
at io.ktor.websocket.DelegatedDefaultWebSocketServerSession.flush(WebSocketServerSession.kt)
at io.ktor.http.cio.websocket.WebSocketSessionKt.close(WebSocketSession.kt:58)
WebSocketSession.close invokes flush that fails because the websocket is already closed at the moment.
Multiple matching paths do not choose higher quality path (component with more static values) when the lower quality one is authenticated
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1158
Ktor Version
1.1.3
Ktor Engine Used(client or server and name)
Jetty server
JVM Version, Operating System and Relevant Context
JDK 11, Linux 5.0
Feedback
Given the following code:
import io.ktor.application.call
import io.ktor.response.respondText
import io.ktor.routing.get
import io.ktor.routing.routing
import io.ktor.server.engine.embeddedServer
import io.ktor.server.jetty.Jetty
fun main() {
val server = embeddedServer(Jetty, port = 9010) {
routing {
get("/foo:{bar}/{baz}") {
call.respondText("Received route1 bar=${call.parameters["bar"]}, baz=${call.parameters["baz"]}")
}
get("/{bar}/{baz}") {
call.respondText("Received route2 bar=${call.parameters["bar"]}, baz=${call.parameters["baz"]}")
}
}
}
server.start(wait = true)
}
The first route has a static component beginning with /foo:
whereas the second route is all dynamic parameters. The first route is therefore of higher quality for matching purposes.
With this call:
http get :9010/foo:a/b
the result is as expected: Received route1 bar=a, baz=b
However, when changing the code to wrap the second route in an authenticate
block like this:
routing {
get("/foo:{bar}/{baz}") {
call.respondText("Received route1 bar=${call.parameters["bar"]}, baz=${call.parameters["baz"]}")
}
authenticate {
get("/{bar}/{baz}") {
call.respondText("Received route2 bar=${call.parameters["bar"]}, baz=${call.parameters["baz"]}")
}
}
}
I would expect the routing decision to be the same, but:
http get :9010/foo:a/b
now attempts to execute the lower quality authenticated path:
HTTP/1.1 401 Unauthorized
Wrong session id get stuck at clients
Currently, if a client keeps wrong/outdated session ID then there is no way to make it forget it. Because of it, a client is sending the wrong ID, again and again, causing a lot of lookup errors at the server. On debugging log-level, it produces a lot of error messages.
Like other servers, we could expire the corresponding cookie
Confusing log message about failed session lookup
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/445
Hello,
I have a problem when I do : call.sessions.get<MySession>() with a wrong header. I have the following error 2018-06-18 15:59:07.681 [nettyCallPool-4-1] DEBUG Application - Failed to lookup session: java.util.NoSuchElementException: No session data found for id 684eb3846f0441113ab51e24471b8062f1d30f5290a11046d17d5725470fdce7.
"For unexistant sessions that are not available, would be like if no session was provided and you can do whatever you need in your route handler"
Thank for your help !
MultiPartData.readAllParts() throws java.io.IOException when multipart list is empty
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/482
Problem description: When receiving an empty multipart-form request on a POST route, the request handler throws an IOException, preventing graceful handling or the display of an error message. Instead, a 500 error message is returned.
Minimum working example:
import io.ktor.application.*
import io.ktor.content.readAllParts
import io.ktor.response.*
import io.ktor.request.*
import io.ktor.routing.post
import io.ktor.routing.routing
fun main(args: Array<String>): Unit = io.ktor.server.netty.DevelopmentEngine.main(args)
fun Application.module() {
routing {
post("/add") {
if(call.request.isMultipart()) {
val multipart = call.receiveMultipart()
val formItems = multipart.readAllParts() // <-- this breaks when sent an empty request
call.respondText("recvd: $multipart")
}
else {
call.respond("no multipart, fix request!")
}
}
}
}
Send a request with an empty multipart body to see the issue:
curl -X POST \
http://localhost:8080/add \
-H 'Cache-Control: no-cache' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW'
Error Message:
22:57:11.866 [nettyCallPool-4-4] ERROR Application - Unhandled: POST - /add
java.io.IOException: Broken delimiter occurred
at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.doResume(Delimited.kt:57)
at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.invoke(Delimited.kt)
at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.invoke(Delimited.kt)
at kotlinx.coroutines.experimental.io.ByteBufferChannel.lookAheadSuspend(ByteBufferChannel.kt:1746)
at kotlinx.coroutines.experimental.io.DelimitedKt.skipDelimiterSuspend(Delimited.kt:55)
at kotlinx.coroutines.experimental.io.DelimitedKt.skipDelimiter(Delimited.kt:50)
at io.ktor.http.cio.MultipartKt.boundary(Multipart.kt:89)
at io.ktor.http.cio.MultipartKt$parseMultipart$1.doResume(Multipart.kt:158)
at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:42)
at kotlinx.coroutines.experimental.DispatchedKt.resumeCancellable(Dispatched.kt:209)
at kotlinx.coroutines.experimental.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:35)
at kotlinx.coroutines.experimental.CoroutineStart.invoke(CoroutineStart.kt:111)
at kotlinx.coroutines.experimental.AbstractCoroutine.start(AbstractCoroutine.kt:165)
at kotlinx.coroutines.experimental.channels.ProduceKt.produce(Produce.kt:95)
at kotlinx.coroutines.experimental.channels.ProduceKt.produce$default(Produce.kt:88)
at io.ktor.http.cio.MultipartKt.parseMultipart(Multipart.kt:145)
at io.ktor.http.cio.MultipartKt.parseMultipart(Multipart.kt:138)
at io.ktor.http.cio.CIOMultipartDataBase.<init>(CIOMultipartData.kt:33)
at io.ktor.http.cio.CIOMultipartDataBase.<init>(CIOMultipartData.kt:31)
at io.ktor.server.engine.DefaultTransformKt.multiPartData(DefaultTransform.kt:70)
at io.ktor.server.engine.DefaultTransformKt.access$multiPartData(DefaultTransform.kt:1)
at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.doResume(DefaultTransform.kt:33)
at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.invoke(DefaultTransform.kt)
at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.invoke(DefaultTransform.kt)
at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
at io.ktor.request.ApplicationReceiveFunctionsKt.receive(ApplicationReceiveFunctions.kt:64)
at io.sebi.ApplicationKt$module$1$1.doResume(application.kt:31)
at io.sebi.ApplicationKt$module$1$1.invoke(application.kt)
at io.sebi.ApplicationKt$module$1$1.invoke(application.kt)
at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
at io.ktor.routing.Routing.executeResult(Routing.kt:100)
at io.ktor.routing.Routing.interceptor(Routing.kt:25)
at io.ktor.routing.Routing$Feature$install$1.doResume(Routing.kt:66)
at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt:51)
at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.doResume(DefaultEnginePipeline.kt:66)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.doResume(NettyApplicationCallHandler.kt:31)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt:10)
at kotlinx.coroutines.experimental.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:44)
at kotlinx.coroutines.experimental.CoroutineStart.invoke(CoroutineStart.kt:113)
at kotlinx.coroutines.experimental.AbstractCoroutine.start(AbstractCoroutine.kt:165)
at kotlinx.coroutines.experimental.BuildersKt__Builders_commonKt.launch(Builders.common.kt:72)
at kotlinx.coroutines.experimental.BuildersKt.launch(Unknown Source)
at kotlinx.coroutines.experimental.BuildersKt__Builders_commonKt.launch$default(Builders.common.kt:64)
at kotlinx.coroutines.experimental.BuildersKt.launch$default(Unknown Source)
at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:22)
at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:16)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38)
at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:844)
Netty: Not started servers leak resources
Netty.create()
does some heavy initialization immediately, for example creating EventLoopGroup
s. These in turn create pipes.
Even though the engine is never started it allocates resources and would only release them when calling stop()
.
This was quite confusing in unit testing where I create engines repeatedly. They may or may not get started and only get stopped if they were actually started.
I'd expect resources to be only allocated when I call start()
, esp. since only stop()
will release them.
repeat(1000) {
embeddedServer(Netty, applicationEngineEnvironment { })
}
Exception in thread "main" java.lang.IllegalStateException: failed to create a child event loop
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:88)
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:58)
at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:52)
at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:96)
at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:91)
at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:72)
at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:52)
at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:190)
at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:90)
at io.ktor.server.netty.Netty.create(Embedded.kt:14)
at io.ktor.server.netty.Netty.create(Embedded.kt:12)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:79)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer$default(EmbeddedServer.kt:77)
at server.MainKt.main(Main.kt:15)
at server.MainKt$$$main.invoke(Unknown Source)
at kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt$createCoroutineUnintercepted$$inlined$createCoroutineFromSuspendFunction$IntrinsicsKt__IntrinsicsJvmKt$1.invokeSuspend(IntrinsicsJvm.kt:205)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlin.coroutines.ContinuationKt.startCoroutine(Continuation.kt:115)
at kotlin.coroutines.jvm.internal.RunSuspendKt.runSuspend(RunSuspend.kt:19)
at server.MainKt.main(Main.kt)
Caused by: io.netty.channel.ChannelException: failed to open a new selector
at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:175)
at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:142)
at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:146)
at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:37)
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:84)
... 19 more
Caused by: java.io.IOException: Too many open files
at java.base/sun.nio.ch.IOUtil.makePipe(Native Method)
at java.base/sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:86)
at java.base/sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:35)
at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:173)
... 23 more
Other
Getting UnsupportedOperationException when calling call.receive
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1249
Ktor pre-release version 1.2.3-rc
When trying to parse the request using:
suspend inline fun <reified T : Any> ApplicationCall.receive(): T = receive(T::class)
In this way:
val obj = call.receive<MyCustomObject>()
It produces de following exception:
java.lang.UnsupportedOperationException: This function has a reified type parameter and thus can only be inlined at compilation time, not called directly.
at kotlin.jvm.internal.Intrinsics.throwUndefinedForReified(Intrinsics.java:193)
at kotlin.jvm.internal.Intrinsics.throwUndefinedForReified(Intrinsics.java:187)
at kotlin.jvm.internal.Intrinsics.reifiedOperationMarker(Intrinsics.java:197)
at "my package"$main$2$3.invokeSuspend(KtorServices.kt:205)
at "my package"$main$2$3.invoke(KtorServices.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
at io.ktor.routing.Routing.executeResult(Routing.kt:147)
at io.ktor.routing.Routing.interceptor(Routing.kt:34)
at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:99)
at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:106)
at io.ktor.features.ContentNegotiation$Feature$install$1.invoke(ContentNegotiation.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:139)
at io.ktor.features.CallLogging$Feature$install$2.invoke(CallLogging.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invokeSuspend(DefaultEnginePipeline.kt:118)
at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:268)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:67)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:141)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:161)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:27)
at io.ktor.server.servlet.KtorServlet$blockingService$1.invokeSuspend(KtorServlet.kt:148)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedKt.resumeCancellable(Dispatched.kt:447)
at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:25)
at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:109)
at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:154)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:53)
at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
at io.ktor.server.servlet.KtorServlet.blockingService(KtorServlet.kt:113)
at io.ktor.server.servlet.KtorServlet.service(KtorServlet.kt:71)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
at com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at com.google.apphosting.runtime.jetty9.ParseBlobUploadHandler.handle(ParseBlobUploadHandler.java:119)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1182)
at com.google.apphosting.runtime.jetty9.AppEngineWebAppContext.doHandle(AppEngineWebAppContext.java:187)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at com.google.apphosting.runtime.jetty9.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:293)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at com.google.apphosting.runtime.jetty9.RpcConnection.handle(RpcConnection.java:213)
at com.google.apphosting.runtime.jetty9.RpcConnector.serviceRequest(RpcConnector.java:81)
at com.google.apphosting.runtime.jetty9.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchServletRequest(JavaRuntime.java:722)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchRequest(JavaRuntime.java:685)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:655)
at com.google.apphosting.runtime.JavaRuntime$NullSandboxRequestRunnable.run(JavaRuntime.java:847)
at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:270)
at java.lang.Thread.run(Thread.java:748)
HttpClient can only be used on the main thread for native targets
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1559
Ktor Version and Engine Used (client or server and name)
Ktor client 1.2.6, default platform engine (iOS)
Describe the bug
HttpClient can only be used from the main thread. It does not matter on which thread the object is instantiated.
The issue is caused by global declarations that are not annotated properly for Kotlin/Native.
Uncaught Kotlin exception: io.ktor.http.URLParserException: Fail to parse url: <redacted>
Caused by: kotlin.native.IncorrectDereferenceException: Trying to access top level value not marked as @ThreadLocal or @SharedImmutable from non-main thread
at 0 test.kexe 0x0000000101edb8e7 kfun:kotlin.Throwable.<init>(kotlin.String?)kotlin.Throwable + 87 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Throwable.kt:22:37)
at 1 test.kexe 0x0000000101ed4ee5 kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
at 2 test.kexe 0x0000000101ed4aa5 kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
at 3 test.kexe 0x0000000101f07b55 kfun:kotlin.native.IncorrectDereferenceException.<init>(kotlin.String)kotlin.native.IncorrectDereferenceException + 85 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/native/Runtime.kt:30:36)
at 4 test.kexe 0x0000000101f25e99 ThrowIncorrectDereferenceException + 137 (/Users/teamcity3/buildAgent/work/4d622a065c544371/runtime/src/main/kotlin/kotlin/native/internal/RuntimeUtils.kt:87:11)
at 5 test.kexe 0x0000000102200ed9 CheckIsMainThread + 25
at 6 test.kexe 0x00000001020e16a3 kfun:io.ktor.http.<get-URL_ALPHABET_CHARS>#internal + 35 (/opt/buildAgent/work/a85294440dc5c6e/ktor-http/common/src/io/ktor/http/Codecs.kt:11:9)
at 7 test.kexe 0x00000001020e2ecd kfun:io.ktor.http.encodeURLPath@kotlin.String.()kotlin.String + 717 (/opt/buildAgent/work/a85294440dc5c6e/ktor-http/common/src/io/ktor/http/Codecs.kt:59:0)
...
I am using the CoroutineWorker library from Autodesk.
To Reproduce
Steps to reproduce the behavior:
- Make a client request from a non-main thread for a native target.
Release 1.4.1
Fix InvalidMutabilityException for tests in master
Native initialisation order leads to fail in constructor when we use this
in field initialization in shared due to freeze:
kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen io.ktor.util.collections.internal.SharedForwardList@9fa908
at kfun:kotlin.Throwable#<init>(kotlin.String?){} (0x2dc9b7)
at kfun:kotlin.Exception#<init>(kotlin.String?){} (0x2d6b65)
at kfun:kotlin.RuntimeException#<init>(kotlin.String?){} (0x2d67c5)
at kfun:kotlin.native.concurrent.InvalidMutabilityException#<init>(kotlin.String){} (0x303585)
at ThrowInvalidMutabilityException (0x304c53)
at MutationCheck (0x5a519e)
at kfun:io.ktor.util.collections.internal.SharedForwardList#<init>(){} (0x4ff407)
at kfun:io.ktor.util.collections.ConcurrentMap#<init>(io.ktor.util.Lock;kotlin.Int){} (0x4e9814)
at kfun:io.ktor.util.collections.ConcurrentMap#<init>(io.ktor.util.Lock?;kotlin.Int;kotlin.Int;kotlin.native.internal.DefaultConstructorMarker?){} (0x4e9af2)
at kfun:io.ktor.util.ConcurrentMapTest#testEmpty(){} (0x52e1a5)
at kfun:io.ktor.util.$ConcurrentMapTest$test$0.$testEmpty$FUNCTION_REFERENCE$14.invoke#internal (0x52f893)
at kfun:io.ktor.util.$ConcurrentMapTest$test$0.$testEmpty$FUNCTION_REFERENCE$14.$<bridge-UNNN>invoke(-1:0){}#internal (0x52f909)
at kfun:kotlin.native.internal.test.BaseClassSuite.TestCase#run(){} (0x32755b)
at kfun:kotlin.native.internal.test.TestRunner.run#internal (0x3215b8)
at kfun:kotlin.native.internal.test.TestRunner.runIteration#internal (0x32276e)
at kfun:kotlin.native.internal.test.TestRunner#run(){}kotlin.Int (0x3230a5)
at kfun:kotlin.native.internal.test#testLauncherEntryPoint(kotlin.Array<kotlin.String>){}kotlin.Int (0x316eec)
at kfun:kotlin.native.internal.test#main(kotlin.Array<kotlin.String>){} (0x316db7)
at Konan_start (0x31701b)
There is also usages of simple references instead of shared in tests and client capabilities.
Empty body in response using macosx64 target
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1622
Ktor Version and Engine Used (client or server and name)
1.3.0, client (macosx64) (using curl)
Describe the bug
When using macosx64 target in a Multiplatform project I'm getting empty BODY in response
HttpClient: REQUEST: http://api.open-notify.org/astros.json
HttpClient: METHOD: HttpMethod(value=GET)
HttpClient: COMMON HEADERS
HttpClient: -> Accept: application/json
HttpClient: -> Accept-Charset: UTF-8
HttpClient: CONTENT HEADERS
HttpClient: BODY Content-Type: null
HttpClient: BODY START
HttpClient: BODY END
The same code for iOS logs
HttpClient: REQUEST: http://api.open-notify.org/astros.json
HttpClient: METHOD: HttpMethod(value=GET)
HttpClient: COMMON HEADERS
HttpClient: -> Accept: application/json
HttpClient: -> Accept-Charset: UTF-8
HttpClient: CONTENT HEADERS
HttpClient: BODY Content-Type: null
HttpClient: BODY START
HttpClient: BODY END
HttpClient: BODY Content-Type: application/json
HttpClient: BODY START
HttpClient: {"people": [{"craft": "ISS", "name": "Andrew Morgan"}, {"craft": "ISS", "name": "Oleg Skripochka"}, {"craft": "ISS", "name": "Jessica Meir"}], "message": "success", "number": 3}
HttpClient: BODY END
The following is the common code that uses ktor
class PeopleInSpaceApi {
private val url = "http://api.open-notify.org/astros.json"
private val client by lazy {
HttpClient() {
install(JsonFeature) {
serializer = KotlinxSerializer(Json(JsonConfiguration(strictMode = false)))
}
install(Logging) {
logger = Logger.DEFAULT
level = LogLevel.ALL
}
}
}
suspend fun fetchPeople(): AstroResult {
return client.get(url)
}
}
I'm using following dependencies
macOSMain.dependencies {
// Coroutines
implementation('org.jetbrains.kotlinx:kotlinx-coroutines-core-macosx64') {
version {
strictly '1.3.3-native-mt'
}
}
// Ktor
implementation "io.ktor:ktor-client-curl:${Versions.ktor}"
implementation "io.ktor:ktor-client-core-macosx64:${Versions.ktor}"
implementation "io.ktor:ktor-client-json-macosx64:${Versions.ktor}"
implementation "io.ktor:ktor-client-logging-macosx64:${Versions.ktor}"
implementation "io.ktor:ktor-client-serialization-macosx64:${Versions.ktor}"
// Serialize
implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime-macosx64:${Versions.kotlinxSerialization}"
}
(kotlinxSerialization = "0.14.0")
MalformedInputException: Input length = 1, when uploading an image multipart
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1766
Ktor Version and Engine Used (client or server and name)
Ktor 1.3.2, Kotlin 1.3.70, Netty server, ContentNegotiation, Jackson parser, Exposed.
I simply used the Ktor project generator for Gradle, and checked a few boxes for modules as mentioned above. After which I managed to connect a database & a few endpoints, but my core endpoint for uploading images doesn't work.
Describe the bug
When trying to upload a simple image using Retrofit & Multipart support in Android, I get a MalformedInputException on my server.
Retrofit/Android code:
@Multipart
@POST("/api/upload")
suspend fun uploadImages(@Part("image") imageFile: RequestBody): UploadResponse
Upload code:
val requestBody: RequestBody = MultipartBody.Builder()
.setType(MultipartBody.FORM)
.addFormDataPart("title", "Image Upload")
.addFormDataPart(
"image", // part name
file.path.split("/").last(), // file name
file.asRequestBody("image/jpg".toMediaType()) // Content-Type: image/jpg
)
.build()
return apiService.uploadImages(requestBody)
Using the new way of creating RequestBody objects in OkHttp. I followed the documentation to receive the multipart data, and I've formed a MalformedInputException.
To Reproduce
Steps to reproduce the behavior:
- Use Retrofit to Upload an image to a Ktor server.
- call
receiveMultipart()
to receive MP on the Ktor side. - See error
Expected behavior
Multipart data should be received correctly.
Stack trace:
[nioEventLoopGroup-4-1] ERROR Application - Unhandled: POST - /api/upload
io.ktor.utils.io.charsets.MalformedInputException: Input length = 1
at io.ktor.utils.io.charsets.CharsetJVMKt.throwExceptionWrapped(CharsetJVM.kt:323)
at io.ktor.utils.io.charsets.CharsetJVMKt.decodeImplSlow(CharsetJVM.kt:289)
at io.ktor.utils.io.charsets.CharsetJVMKt.decodeExactBytes(CharsetJVM.kt:254)
at io.ktor.utils.io.core.StringsKt.readTextExactBytes(Strings.kt:293)
at io.ktor.utils.io.core.StringsKt.readTextExactBytes$default(Strings.kt:292)
at io.ktor.utils.io.core.AbstractInput.readText(AbstractInput.kt:468)
at io.ktor.utils.io.core.AbstractInput.readText$default(AbstractInput.kt:465)
at io.ktor.http.cio.CIOMultipartDataBase.partToData(CIOMultipartData.kt:127)
at io.ktor.http.cio.CIOMultipartDataBase$partToData$1.invokeSuspend(CIOMultipartData.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:844)
Ktor 1.3.1 Fails File Upload with MalformedInputException
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1845
Hi. I am using Ktor Client with version 1.3.1 for MPP
I have been using Ktor version 1.2.6 in my MPP project with Kotlin version 1.3.61 and it was working fine, so I was able to upload files without a problem. Later I update Kotlin version to 1.3.72 which forced me to update Ktor version 1.3.1 as version 1.2.6 was compiled with Kotlin version 1.3.61 which IDE can not compile with Kotlin plugin 1.3.72. After that file upload call started to give me the following exception.
2020-05-02 17:35:35.186 14564-14671/com.inivo.client.debug E/AndroidRuntime: FATAL EXCEPTION: ktor-android-dispatcher-worker-2
Process: com.inivo.client.debug, PID: 14564
io.ktor.utils.io.charsets.MalformedInputException: Input length = 1
at io.ktor.utils.io.charsets.CharsetJVMKt.throwExceptionWrapped(CharsetJVM.kt:323)
at io.ktor.utils.io.charsets.CharsetJVMKt.decode(CharsetJVM.kt:199)
at io.ktor.utils.io.charsets.EncodingKt.decode(Encoding.kt:101)
at io.ktor.utils.io.core.StringsKt.readText(Strings.kt:251)
at io.ktor.utils.io.core.StringsKt.readText$default(Strings.kt:250)
at io.ktor.client.features.logging.Logging$logRequestBody$2$1.invokeSuspend(Logging.kt:111)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
at kotlinx.coroutines.EventLoop.processUnconfinedEvent(EventLoop.common.kt:69)
at kotlinx.coroutines.DispatchedTaskKt.resumeUnconfined(DispatchedTask.kt:184)
at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:108)
at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:307)
at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:317)
at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:249)
at io.ktor.utils.io.ByteBufferChannel.resumeWriteOp(ByteBufferChannel.kt:2210)
at io.ktor.utils.io.ByteBufferChannel.bytesRead(ByteBufferChannel.kt:930)
at io.ktor.utils.io.ByteBufferChannel.readAsMuchAsPossible(ByteBufferChannel.kt:544)
at io.ktor.utils.io.ByteBufferChannel.readAvailable$suspendImpl(ByteBufferChannel.kt:606)
at io.ktor.utils.io.ByteBufferChannel.readAvailable(Unknown Source:0)
at io.ktor.utils.io.jvm.javaio.WritingKt.copyTo(Writing.kt:21)
at io.ktor.utils.io.jvm.javaio.WritingKt.copyTo$default(Writing.kt:12)
at io.ktor.client.engine.android.AndroidClientEngineKt.writeTo(AndroidClientEngine.kt:112)
at io.ktor.client.engine.android.AndroidClientEngine.execute(AndroidClientEngine.kt:79)
at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:83)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
I am uploading a file like this.
override suspend fun sendImageAsync(
token: String,
file: File,
fileType: String,
id: Long
): Unit = client.request {
fullUrl("${RestApi.SEND_IMAGE}/${id}")
method = HttpMethod.Post
body = MultiPartFormDataContent(
formData {
append(
"image",
file.fileData.bytes,
headers = Headers.build {
append(HttpHeaders.ContentType, "image/jpg")
append(
HttpHeaders.ContentDisposition,
"filename=${file.name}"
)
append("Connection", "Close")
}
)
}
)
header(HttpHeaders.Connection, "close")
header(HttpHeaders.Authorization, makeAuthToken(token))
}
Here file data is just an object which keeps File data as a byte array so by that I can convert image to byte array (either from Android / IOS) and upload it.
Thank you very much for your help in advanced.
ktor client does not log request body
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1729
Ktor Version and Engine Used (client or server and name)
ver: 1.3.1, 1.3.2
engine: Apache
app: client
dependencies:
dependencies {
implementation(kotlin("stdlib-jdk8"))
implementation(platform("io.ktor:ktor-bom:1.3.2"))
implementation("io.ktor:ktor-client-core")
implementation("io.ktor:ktor-client-logging-jvm")
implementation("io.ktor:ktor-client-apache")
implementation("io.ktor:ktor-client-json-jvm")
implementation("org.slf4j:slf4j-simple:1.7.26")
implementation("org.apache.httpcomponents:httpasyncclient:4.1.4")
}
Describe the bug
Request body is not logged even with LogLevel.ALL
To Reproduce
package com.example
import io.ktor.client.HttpClient
import io.ktor.client.engine.apache.Apache
import io.ktor.client.features.logging.DEFAULT
import io.ktor.client.features.logging.LogLevel
import io.ktor.client.features.logging.Logger
import io.ktor.client.features.logging.Logging
import io.ktor.client.request.post
import io.ktor.util.KtorExperimentalAPI
import kotlinx.coroutines.runBlocking
@KtorExperimentalAPI
fun main() {
runBlocking {
val client = HttpClient(Apache) {
install(Logging) {
logger = Logger.DEFAULT
level = LogLevel.ALL
}
}
client.post<String>("http://localhost:8080") {
body = """{ "test": "testVal" }"""
}
}
}
Expected behavior
Request body is in the log
Actual result
[main] INFO io.ktor.client.HttpClient - REQUEST: http://localhost:8080/
[main] INFO io.ktor.client.HttpClient - METHOD: HttpMethod(value=POST)
[main] INFO io.ktor.client.HttpClient - COMMON HEADERS
[main] INFO io.ktor.client.HttpClient - -> Accept-Charset: UTF-8
[main] INFO io.ktor.client.HttpClient - -> Accept: */*
[main] INFO io.ktor.client.HttpClient - CONTENT HEADERS
[main] INFO io.ktor.client.HttpClient - BODY Content-Type: text/plain; charset=UTF-8
[main] INFO io.ktor.client.HttpClient - RESPONSE: 200
--- remaining response stuff skipped for brevity ---
kotlin.native.concurrent.InvalidMutabilityException with 1.3.3-native-mt
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1543
KTOR native with the new multi threaded coroutines fails with InvalidMutabilityException.
coroutines version: 1.3.3-native-mt
ktor version: 1.3.0-rc2
Here is the related code:
suspend fun doFetch() {
HttpClient().use {
val htmlContent = it.get<String>("https://en.wikipedia.org/wiki/Main_Page")
println(htmlContent)
}
}
fun doesNotWork() {
runBlocking(Dispatchers.Default) { // remove Dispatchers.Default and it works
doFetch()
}
}
fun main() {
doesNotWork()
}
to reproduce, checkout the following project:
https://github.com/yigit/ktor-coroutines-bug
run ./gradlew runDELX --stacktrace
from command line.
It fails with :
Uncaught Kotlin exception: kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen io.ktor.utils.io.ByteChannelNative@c006ba18
at kfun:kotlin.Throwable.<init>(kotlin.String?)kotlin.Throwable (0x2d9607)
at kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception (0x2d3115)
at kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException (0x2d2cd5)
at kfun:kotlin.native.concurrent.InvalidMutabilityException.<init>(kotlin.String)kotlin.native.concurrent.InvalidMutabilityException (0x2fff65)
at ThrowInvalidMutabilityException (0x301673)
at MutationCheck (0x5a742e)
at kfun:io.ktor.utils.io.ByteChannelSequentialBase.<set-waitingForRead>#internal (0x403fed)
at kfun:io.ktor.utils.io.ByteChannelSequentialBase.$awaitSuspendCOROUTINE$73.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? (0x417d9f)
at kfun:io.ktor.utils.io.ByteChannelSequentialBase.awaitSuspend(kotlin.Int)ValueType (0x418244)
at kfun:io.ktor.utils.io.ByteChannelSequentialBase.$readRemainingSuspendCOROUTINE$60.invokeSuspend#internal (0x40f22c)
at kfun:io.ktor.utils.io.ByteChannelSequentialBase.readRemainingSuspend#internal (0x40f5c0)
at kfun:io.ktor.utils.io.ByteChannelSequentialBase.$readRemainingCOROUTINE$59.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? (0x40e936)
at kfun:io.ktor.utils.io.ByteChannelSequentialBase.readRemaining(kotlin.Long;kotlin.Int)io.ktor.utils.io.core.ByteReadPacket (0x40ec6e)
at kfun:io.ktor.utils.io.readRemaining@io.ktor.utils.io.ByteReadChannel.()io.ktor.utils.io.core.ByteReadPacket (0x421354)
at kfun:io.ktor.client.features.HttpPlainText.Feature.$install$lambda-1COROUTINE$26.invokeSuspend#internal (0x53c9c4)
at kfun:io.ktor.client.features.HttpPlainText.Feature.$install$lambda-1COROUTINE$26.invoke#internal (0x53d3ef)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x4d3e3d)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal (0x4d34be)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceedWith#internal (0x4d3697)
at kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-1COROUTINE$22.invokeSuspend#internal (0x538612)
at kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-1COROUTINE$22.invoke#internal (0x538baf)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x4d3e3d)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal (0x4d34be)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.execute#internal (0x4d38c1)
at kfun:io.ktor.util.pipeline.Pipeline.execute(#GENERIC_kotlin.Any;#GENERIC_kotlin.Any)#GENERIC_kotlin.Any (0x4ce690)
at kfun:io.ktor.client.call.HttpClientCall.$receiveCOROUTINE$12.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? (0x5205d3)
at kfun:io.ktor.client.call.HttpClientCall.receive(io.ktor.client.call.TypeInfo)kotlin.Any (0x520c06)
at kfun:$doFetchCOROUTINE$0.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? (0x565423)
at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0x2f4968)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x4d4508)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x4d4072)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x4d5d9c)
at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0x2f4c49)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x4d4508)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x4d4072)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x4d5d9c)
at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0x2f4c49)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith#internal (0x4d4508)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x4d4072)
at kfun:io.ktor.util.pipeline.SuspendFunctionGun.object-1.resumeWith#internal (0x4d5d9c)
at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0x2f4c49)
at kfun:kotlinx.coroutines.DispatchedTask.run() (0x3e2fca)
at kfun:kotlinx.coroutines.EventLoopImplBase.processNextEvent()ValueType (0x3bb507)
at kfun:kotlinx.coroutines.runEventLoop$kotlinx-coroutines-core(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>) (0x3f665c)
at kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.start$lambda-0#internal (0x3fe271)
at kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$80.invoke#internal (0x3fe3db)
at kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$80.$<bridge-UNN>invoke()#internal (0x3fe43b)
at WorkerLaunchpad (0x30129c)
at _ZN6Worker19processQueueElementEb (0x5ac1c9)
at _ZN12_GLOBAL__N_113workerRoutineEPv (0x5aca2d)
at (0x7f54c8681c73)
at clone (0x7f54c83b3def)
at ((nil))
kotlinMultiplatform publications miss the gradle metadata
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1619
Checked on releases 1.3.0 and 1.3.1.
This makes these publications plain useless.
kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen io.ktor.client.request.HttpRequestPipeline@545e18f8
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/887
Ktor Version
implementation("io.ktor:ktor-client-ios:1.1.1")
Ktor Engine Used(client or server and name)
Client
JVM Version, Operating System and Relevant Context
MBP 2017 JVM 1.8
Feedback
This exception was printed out,and the test can't print out the expected result
suspend fun getDemoJSONDataFromNet(): String {
val result = client.get<String>("https://en.wikipedia.org/wiki/Main_Page")
Logger.d(tag, "the return data is $result")
return result
}
fun getDataIos(callBack: (result: String) -> Unit) {
runBlocking {
val job = async {
val result = HttpUtil.getDemoJSONDataFromNet()
Logger.d("SimpleIos","result value is $result")
callBack(result)
}
job.await()
}
}
@Test
fun testGetData() {
getDataIos {
println(it)
}
}
Exception Log:
Instances of kotlin.Error, kotlin.RuntimeException and subclasses aren't propagated from Kotlin to Objective-C/Swift.
Other exceptions can be propagated as NSError if method has or inherits @Throws annotation.
Uncaught Kotlin exception: kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen io.ktor.client.request.HttpRequestPipeline@2ef2a08
at 0 app 0x000000010dba2896 kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception + 70
at 1 app 0x000000010dba27b6 kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException + 70
at 2 app 0x000000010dba40e6 kfun:kotlin.native.concurrent.InvalidMutabilityException.<init>(kotlin.String)kotlin.native.concurrent.InvalidMutabilityException + 70
at 3 app 0x000000010dc3c6b8 ThrowInvalidMutabilityException + 280
at 4 app 0x000000010dc5ee48 MutationCheck + 24
at 5 app 0x000000010dce8a83 kfun:io.ktor.util.pipeline.Pipeline.<set-interceptors>#internal + 67
at 6 app 0x000000010dce86a6 kfun:io.ktor.util.pipeline.Pipeline.notSharedInterceptorsList#internal + 70
at 7 app 0x000000010dce84d6 kfun:io.ktor.util.pipeline.Pipeline.cacheInterceptors#internal + 710
at 8 app 0x000000010dce6292 kfun:io.ktor.util.pipeline.Pipeline.sharedInterceptorsList#internal + 98
at 9 app 0x000000010dce61ca kfun:io.ktor.util.pipeline.Pipeline.createContext$ktor-utils(#GENERIC_kotlin.Any;#GENERIC_kotlin.Any)io.ktor.util.pipeline.PipelineExecutor<#GENERIC_kotlin.Any> + 90
at 10 app 0x000000010dce60f0 kfun:io.ktor.util.pipeline.Pipeline.execute(#GENERIC_kotlin.Any;#GENERIC_kotlin.Any)#GENERIC_kotlin.Any + 96
at 11 app 0x000000010dd313a8 kfun:io.ktor.client.HttpClient.$execute$COROUTINE$1.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 296
at 12 app 0x000000010dd31522 kfun:io.ktor.client.HttpClient.execute(io.ktor.client.request.HttpRequestBuilder)io.ktor.client.call.HttpClientCall + 130
at 13 app 0x000000010dd32696 kfun:io.ktor.client.call.$call$COROUTINE$3.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 758
at 14 app 0x000000010dd32852 kfun:io.ktor.client.call.call@io.ktor.client.HttpClient.(kotlin.coroutines.SuspendFunction1<io.ktor.client.request.HttpRequestBuilder,kotlin.Unit>)io.ktor.client.call.HttpClientCall + 130
at 15 app 0x000000010dd330eb kfun:io.ktor.client.call.call@io.ktor.client.HttpClient.(io.ktor.client.request.HttpRequestBuilder)io.ktor.client.call.HttpClientCall + 123
at 16 app 0x000000010db97f3f kfun:sample.HttpUtil.$getDemoJSONDataFromNet$COROUTINE$0.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 1551
at 17 app 0x000000010db97821 kfun:sample.HttpUtil.getDemoJSONDataFromNet()kotlin.String + 113
at 18 app 0x000000010dba0de9 kfun:sample.$getDataIos$lambda-1$lambda-0$COROUTINE$3.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 313
at 19 app 0x000000010dbd4801 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 385
at 20 app 0x000000010dca7b89 kfun:kotlinx.coroutines.DispatchedTask.run() + 1129
at 21 app 0x000000010dca92d6 kfun:kotlinx.coroutines.EventLoopImpl.processNextEvent()ValueType + 422
at 22 app 0x000000010dccf7f7 kfun:kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal + 263
at 23 app 0x000000010dccf5d7 kfun:kotlinx.coroutines.runBlocking(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,#GENERIC>)Generic + 983
at 24 app 0x000000010dccfb4c kfun:kotlinx.coroutines.runBlocking$default(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,#GENERIC>;kotlin.Int)Generic + 204
at 25 app 0x000000010dba06e4 kfun:sample.getDataIos(kotlin.Function1<kotlin.String,kotlin.Unit>) + 100
at 26 app 0x000000010dba0618 kfun:sample.main(kotlin.Array<kotlin.String>) + 216
at 27 iosApp 0x000000010d897774 $S6iosApp14ViewControllerC11viewDidLoadyyFTo + 36
at 28 UIKitCore 0x0000000112ea84e1 -[UIViewController loadViewIfRequired] + 1186
MPP JS client: Can't resolve 'text-encoding'
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/961
Ktor Version
1.1.2
Ktor Engine Used(client or server and name)
Ktor client with default engine
JVM Version, Operating System and Relevant Context
JDK 1.8, Windows, Kotlin JS
Feedback
We are using the ktor client in a multiplatform project (android, iOS and web). Starting with version 1.1.0 the build of the JS client fails with the following message:
ERROR in ./kotlinx-io.js
Module not found: Error: Can't resolve 'text-encoding' in '<mypath>\kotlinmultiplatform\web\build\kotlin-js-min\main'
@ ./kotlinx-io.js 5309:21-45 5320:21-45
@ ./ktor-utils.js
@ ./web.js
If I manually add text-encoding
as a npm-dependeny, it works.
If I list the npm dependency tree with the previous working ktor version 1.0.1 I get:
+-- kotlinx-io@0.1.4 -> <mypath>\kotlinmultiplatform\web\build\node_modules_imported\kotlinx-io extraneous
+-- kotlinx-io-js@0.1.1 -> <mypath>\kotlinmultiplatform\web\build\node_modules_imported\kotlinx-io-js
| +-- kotlin@1.3.10
| +-- kotlin-test@1.3.10
| | `-- kotlin@1.3.10 deduped
| +-- kotlinx-atomicfu@0.11.12 -> <mypath>\kotlinmultiplatform\web\build\node_modules_imported\kotlinx-atomicfu deduped
| `-- text-encoding@0.7.0
If I list the npm dependency tree with a ktor version >= 1.1.0 the kotlinx-io-js
(and of course text-encoding
) is missing
Error Ktor running on background thread on iOS
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1538
Ktor Version and Engine Used (client or server and name)
Ktor client version 1.3.0-rc2
build.gradle:
ext.ktor_version = '1.3.0-rc2'
ext.kotlin_coroutines_version = '1.3.3-native-mt'
sourceSets {
commonMain {
dependencies {
implementation kotlin('stdlib-common')
implementation ("io.ktor:ktor-client-core:$ktor_version") {
exclude group: "org.jetbrains.kotlinx", module: "kotlinx-coroutines-core-common"
}
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core-common:$kotlin_coroutines_version"
}
}
iosMain {
dependencies {
implementation ("io.ktor:ktor-client-ios:$ktor_version") {
exclude group: "org.jetbrains.kotlinx", module: "kotlinx-coroutines-core-native"
exclude group: "org.jetbrains.kotlinx", module: "kotlinx-coroutines-core-common"
}
implementation ("io.ktor:ktor-client-core-native:$ktor_version") {
exclude group: "org.jetbrains.kotlinx", module: "kotlinx-coroutines-core-native"
exclude group: "org.jetbrains.kotlinx", module: "kotlinx-coroutines-core-common"
}
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core-native:$kotlin_coroutines_version"
}
}
}
Describe the bug
Executing an API request on the background thread results in kotlin.native.concurrent.InvalidMutabilityException
Full stacktrace
kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen io.ktor.client.request.HttpRequestPipeline@b5b748
at 0 app 0x0000000105688a87 kfun:kotlin.Throwable.<init>(kotlin.String?)kotlin.Throwable + 87
at 1 app 0x0000000105682215 kfun:kotlin.Exception.<init>(kotlin.String?)kotlin.Exception + 85
at 2 app 0x0000000105681dd5 kfun:kotlin.RuntimeException.<init>(kotlin.String?)kotlin.RuntimeException + 85
at 3 app 0x00000001056b6005 kfun:kotlin.native.concurrent.InvalidMutabilityException.<init>(kotlin.String)kotlin.native.concurrent.InvalidMutabilityException + 85
at 4 app 0x00000001056b77d8 ThrowInvalidMutabilityException + 680
at 5 app 0x00000001059a8c58 MutationCheck + 104
at 6 app 0x00000001058c9400 kfun:io.ktor.util.pipeline.Pipeline.<set-interceptors>#internal + 96
at 7 app 0x00000001058c98f5 kfun:io.ktor.util.pipeline.Pipeline.notSharedInterceptorsList#internal + 85
at 8 app 0x00000001058c8a8a kfun:io.ktor.util.pipeline.Pipeline.cacheInterceptors#internal + 650
at 9 app 0x00000001058c97a4 kfun:io.ktor.util.pipeline.Pipeline.sharedInterceptorsList#internal + 340
at 10 app 0x00000001058c5f57 kfun:io.ktor.util.pipeline.Pipeline.createContext$ktor-utils(#GENERIC_kotlin.Any;#GENERIC_kotlin.Any)io.ktor.util.pipeline.PipelineExecutor<#GENERIC_kotlin.Any> + 263
at 11 app 0x00000001058c5d71 kfun:io.ktor.util.pipeline.Pipeline.execute(#GENERIC_kotlin.Any;#GENERIC_kotlin.Any)#GENERIC_kotlin.Any + 273
at 12 app 0x000000010590797a kfun:io.ktor.client.HttpClient.$executeCOROUTINE$6.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 666
at 13 app 0x0000000105907cc4 kfun:io.ktor.client.HttpClient.execute(io.ktor.client.request.HttpRequestBuilder)io.ktor.client.call.HttpClientCall + 308
at 14 app 0x000000010593bb7c kfun:io.ktor.client.statement.HttpStatement.$executeUnsafeCOROUTINE$35.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? + 956
at 15 app 0x000000010593be8b kfun:io.ktor.client.statement.HttpStatement.executeUnsafe$ktor-client-core()io.ktor.client.statement.HttpResponse + 235
at 16 app 0x000000010594b557 kfun:sample.ClientApi.$getQuote$lambda-0COROUTINE$0.invokeSuspend#internal + 6775
at 17 app 0x00000001056a9ec8 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) + 712
at 18 app 0x00000001057ca07c kfun:kotlinx.coroutines.DispatchedTask.run() + 2780
at 19 app 0x000000010577fa18 kfun:kotlinx.coroutines.EventLoopImplBase.processNextEvent()ValueType + 792
at 20 app 0x00000001057e3e31 kfun:kotlinx.coroutines.runEventLoop$kotlinx-coroutines-core(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>) + 881
at 21 app 0x00000001057eb412 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.start$lambda-0#internal + 402
at 22 app 0x00000001057eb5fb kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$71.invoke#internal + 59
at 23 app 0x00000001057eb65b kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$71.$<bridge-UNN>invoke()#internal + 59
at 24 app 0x00000001056b73a1 WorkerLaunchpad + 177
at 25 app 0x00000001059b1909 _ZN6Worker19processQueueElementEb + 2569
at 26 app 0x00000001059b1eb6 _ZN12_GLOBAL__N_113workerRoutineEPv + 54
at 27 libsystem_pthread.dylib 0x00007fff51bfe2eb _pthread_body + 126
at 28 libsystem_pthread.dylib 0x00007fff51c01249 _pthread_start + 66
at 29 libsystem_pthread.dylib 0x00007fff51bfd40d thread_start + 13
To Reproduce
Steps to reproduce the behavior:
- Clone sample app from https://github.com/aleksey-chugaev/ktor-mt-app
- Run iosApp in emulator
- See error
Expected behavior
Should successfully execute the request.
Bump Netty version to mitigate CVE-2020-11612
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1778
As per https://nvd.nist.gov/vuln/detail/CVE-2020-11612, there is a known vulnerability with the current Netty version used by ktor - 4.1.44.Final
I'll raise a PR to bump to 4.1.48.Final
Ktor ssl websocket doesn't work on Android
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/732
Hi, I've used latest ktor on android, but got exception when used ssl websocket.
private val client = HttpClient(CIO).config {
install(WebSockets)
}
client.wss(host = uri.host, port = uri.port, path = uri.encodedPath) { process(this) }
Fatal Exception: java.lang.NoSuchMethodError
No static method getInstanceStrong()Ljava/security/SecureRandom; in class Ljava/security/SecureRandom; or its super classes (declaration of 'java.security.SecureRandom' appears in /system/framework/core-libart.jar)
Fatal Exception: java.lang.NoSuchMethodError: No static method getInstanceStrong()Ljava/security/SecureRandom; in class Ljava/security/SecureRandom; or its super classes (declaration of 'java.security.SecureRandom' appears in /system/framework/core-libart.jar)
at io.ktor.client.engine.cio.CIOEngineConfigKt.<clinit>(CIOEngineConfig.kt:9)
at io.ktor.client.engine.cio.HttpsConfig.<init>(CIOEngineConfig.kt:29)
at io.ktor.client.engine.cio.CIOEngineConfig.<init>(CIOEngineConfig.kt:13)
at io.ktor.client.engine.cio.CIO.create(CIO.kt:14)
at io.ktor.client.HttpClientKt.HttpClient(HttpClient.kt:27)
at io.ktor.client.HttpClientKt.HttpClient$default(HttpClient.kt:24)
at com.eld.common.telemetry.TelemetryService.<init>(TelemetryService.kt:97)
at com.eld.android.ApplicationService.initialize(ApplicationService.kt:335)
at com.eld.android.ApplicationService.start(ApplicationService.kt:452)
at com.eld.android.ApplicationService.onStartCommand(ApplicationService.kt:435)
at android.app.ActivityThread.handleServiceArgs(ActivityThread.java:3014)
at android.app.ActivityThread.-wrap17(ActivityThread.java)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1442)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:148)
at android.app.ActivityThread.main(ActivityThread.java:5421)
at java.lang.reflect.Method.invoke(Method.java)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
Websocket failures
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/752
Ktor Version
0.9.3
Ktor Engine Used(client or server and name)
Server Netty
JVM Version, Operating System and Relevant Context
1.8.0_172
Feedback
I found various issues with Websockets, often happening after they run for a long time (months)
They work properly but all of a sudden some exception happen and subsequent websocket clients are not able to connect again, websocket handlers are never called again.
Exception i observed throughout time are below:
This seems to be some sort of a protocol error or incomplete packet received.
After this happened, it went into an inconsistent state.
2018-11-19 14:39:46.485|nettyWorkerPool-3-2 @coroutine#82620|Routing.kt:121:doResume() - [ERROR] Application: Websocket handler failed
kotlinx.coroutines.experimental.JobCancellationException: Child job was cancelled because of parent failure
at kotlinx.coroutines.experimental.JobSupport.cancelChildrenInternal(JobSupport.kt:587)
at kotlinx.coroutines.experimental.JobSupport.makeCompletingInternal(JobSupport.kt:572)
at kotlinx.coroutines.experimental.JobSupport.makeCompleting$kotlinx_coroutines_core(JobSupport.kt:525)
at kotlinx.coroutines.experimental.JobSupport.makeCompletingOnCancel(JobSupport.kt:519)
at kotlinx.coroutines.experimental.JobSupport.cancel(JobSupport.kt:463)
at kotlinx.coroutines.experimental.CoroutineExceptionHandlerKt.handleCoroutineException(CoroutineExceptionHandler.kt:45)
at kotlinx.coroutines.experimental.StandaloneCoroutine.onFinishingInternal$kotlinx_coroutines_core(Builders.common.kt:174)
at kotlinx.coroutines.experimental.JobSupport.makeCompletingInternal(JobSupport.kt:577)
at kotlinx.coroutines.experimental.JobSupport.makeCompletingOnce$kotlinx_coroutines_core(JobSupport.kt:539)
at kotlinx.coroutines.experimental.AbstractCoroutine.resumeWithException(AbstractCoroutine.kt:122)
at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resumeWithException(CoroutineImpl.kt:47)
at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:41)
at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:41)
at kotlinx.coroutines.experimental.DispatchedTask$DefaultImpls.run(Dispatched.kt:162)
at kotlinx.coroutines.experimental.io.internal.MutableDelegateContinuation.run(MutableDelegateContinuation.kt:14)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Unsupported opcode b
at io.ktor.http.cio.websocket.FrameParser.getFrameType(FrameParser.kt:26)
at io.ktor.http.cio.websocket.FrameParser.parseHeader1(FrameParser.kt:69)
at io.ktor.http.cio.websocket.FrameParser.handleStep(FrameParser.kt:56)
at io.ktor.http.cio.websocket.FrameParser.frame(FrameParser.kt:52)
at io.ktor.http.cio.websocket.WebSocketReader.parseLoop(WebSocketReader.kt:72)
at io.ktor.http.cio.websocket.WebSocketReader.readLoop(WebSocketReader.kt:63)
at io.ktor.http.cio.websocket.WebSocketReader$readLoop$1.doResume(WebSocketReader.kt)
at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:42)
... 9 common frames omitted
This one as well, how is possible to send the same response over a websocket?
It's not quite clear.
2018-11-21 07:59:43.079|nettyWorkerPool-3-2 @coroutine#182725|Routing.kt:121:doResume() - [ERROR] Application: Websocket handler failed
io.ktor.server.engine.BaseApplicationResponse$ResponseAlreadySentException: Response has already been sent
at io.ktor.server.engine.BaseApplicationResponse$$special$$inlined$apply$lambda$1.doResume(BaseApplicationResponse.kt:32)
at io.ktor.server.engine.BaseApplicationResponse$$special$$inlined$apply$lambda$1.invoke(BaseApplicationResponse.kt)
at io.ktor.server.engine.BaseApplicationResponse$$special$$inlined$apply$lambda$1.invoke(BaseApplicationResponse.kt:16)
at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
at io.ktor.pipeline.PipelineContext.proceedWith(PipelineContext.kt:35)
at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.doResume(DefaultTransform.kt:22)
at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$1.invoke(DefaultTransform.kt)
at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
at routes.AdapterKt$handleAfClients$1.doResume(Adapter.kt:143)
at routes.AdapterKt$handleAfClients$1.invoke(Adapter.kt)
at routes.AdapterKt$handleAfClients$1.invoke(Adapter.kt)
at io.ktor.websocket.RoutingKt$proceedWebSocket$2.doResume(Routing.kt:119)
at io.ktor.websocket.RoutingKt$proceedWebSocket$2.invoke(Routing.kt)
at io.ktor.websocket.RoutingKt$proceedWebSocket$2.invoke(Routing.kt)
at io.ktor.http.cio.websocket.DefaultWebSocketSessionImplKt.run(DefaultWebSocketSessionImpl.kt:165)
at io.ktor.websocket.RoutingKt.proceedWebSocket(Routing.kt:117)
at io.ktor.websocket.RoutingKt$webSocket$2.doResume(Routing.kt:92)
at io.ktor.websocket.RoutingKt$webSocket$2.invoke(Routing.kt)
at io.ktor.websocket.RoutingKt$webSocket$2.invoke(Routing.kt)
at io.ktor.websocket.RoutingKt$webSocketRaw$2$1$1$1$1.doResume(Routing.kt:52)
at io.ktor.websocket.RoutingKt$webSocketRaw$2$1$1$1$1.invoke(Routing.kt)
at io.ktor.websocket.RoutingKt$webSocketRaw$2$1$1$1$1.invoke(Routing.kt)
at io.ktor.http.cio.websocket.RawWebSocketKt.start(RawWebSocket.kt:49)
at io.ktor.websocket.WebSocketUpgrade$upgrade$2.doResume(WebSocketUpgrade.kt:54)
at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:42)
at kotlinx.coroutines.experimental.DispatchedTask$DefaultImpls.run(Dispatched.kt:162)
at kotlinx.coroutines.experimental.DispatchedContinuation.run(Dispatched.kt:26)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
ktor-client: Proxy support
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1245
I'm developing an application with Kotlin and ktor (as client), but haven't found anything about working with any kind of proxy, especially SOCKS one.
Client MPP Socket Support
This issue was imported from GitHub issue: https://github.com/ktorio/ktor/issues/1454
Subsystem
Common Client
Is your feature request related to a problem? Please describe.
https://github.com/ktorio/ktor/issues/1121
Describe the solution you'd like
Has there been any development on the common support for sockets? The issue above was added to 1.3.0 milestone, but this has not been added to the patch notes nor is in the beta.
Motivation to include to ktor
Client sockets will help communicate with the raw sockets from server as well as external APIs which use sockets for voice or audio.